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I. INTRODUCTION 



This thesis deals with detection and estimation using spatially separated sensors. 
A typical practical situation is a surveillance system [1] in which a large number of 
sensors monitor some region of space, earth or sea and report their findings to a global 
processor. The sensors themselves may use thermal, acoustic or infrared effects to form 
their observations. The global processor performs some processing on the data to come 
with a decision or for taking actions. Because of many considerations such as 
bandwidth communication limitations, time delay or because the amount of 
information is too massive to be processed by a single processor, the processing is 
carried out on many levels. As an example consider the case of distributed detection. 
Detection is performed at the sensor level and at the fusion center. 

Due to the loss of information in the local processing, the overall performance 
degrades. However a great communication bandwidth reduction results. If the 
communication channels can support more information flow, then it is wise to perform 
"softer - ' processing at the local level, to send more information to the fusion center, and 
to use the information available there effectively. 

The purpose of this chapter is to define the Distributed Signal Processing (DSP) 
problem in general and to show some reasons and situations in which it replaces 
Centralized Signal Processing (CSP) techniques. We then will review the status of the 
research on Decentralized Detection (DD) problem, one of the basic problems of DSP. 
Finally the contributions and organization of this thesis are described. 

A. OVERVIEW 

Classical (Centralized) Signal Processing (CSP) assumes complete availability of 
all information (signals) at one central processor for processing (decision making, 
computing, detection, estimation, etc...). While this situation is realistic in some cases, 
many real world systems are too large for the classical processing to be practically 
applied. Power systems, detection networks, large manufacturing systems and military 
organizations are among those systems in which total centralized signal processing is 
hard to apply. Some of the reasons and considerations for the limations of CSP are 
[ 2 , 3 ]: 
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1. In large systems, each processor has partial information of some credibility. 
While total information is distributed in the whole system, total centeralization 
of the information at one processor is impractical, inconvenient or expensive 
due to limitations in the system's communication channels, memory or 
computation and information capabilities. 

2. In some cases, processing speed is a bottleneck. Increasing local processing of 
the data at each processor and sending processed data to the next level of 
processors will help relieve the problem. 

3. When reliability of the system is of major concern, distributed processing may 
better tolerate various kinds of equipment failures. Less complex centralized 
processing is more easily shifted to a new location. 

4. In cases when security is a major problem, increasing local processing will 
decrease the information handled between the processors, so limit any other 
system's access to the process. 

5. As the cost of computation has decreased dramatically relative to the cost of 
communication, it is advantageous to trade off increased computation for 
reduced communication. So in Distributed Sensor Networks (DSN) involving 
geographically distributed sensors that collect data, it may be more economical 
to locally process the data and send condensed summaries to other processors. 

Distributed Signal Processing (DSP), in contrast to CSP, has several processors 
that cooperate together to best achieve a global task according to some criterion. A 
basic problem in DSP, which has attracted much attention recently, is the 
Decentralized Detection (DD) problem (hypothesis testing). The DD problem will be 
a major concern in this thesis. A summary of its status is given in the following section. 

B. MOTIVATION 

There has been an increased interest in the DD problem since Tenney and 
Sandell introduced it in 1981 [5]. They extended the classical Bayesian formulation of 
the detection problem to distributed environments. Because their work was the 
pioneering one in DD and because we will refer to it often in this thesis , let us 
consider it now in some detail together with the Centralized Detection (CD) problem. 
Also, because detection is dealt with throughout a large portion of this thesis, we will 
make some remarks about the phenomena to be detected and about detection criterion. 
The Phenomena 

Consider observing a phenomena H of M possible states in order to determine 
which of them is true. For M = 2, the state H Q is called the null hypothesis and Hj 
the alternative hypothesis. Their probabilities of occurrence 

P(H 0 )=P 0 , P(H 1 )=P 1 (1.1) 
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are assumed to be known. 

The Sensor Observations 

The phenomena H is observed by N sensors Sj,S 2 ,...,S N . The sensor 
observations are ypy 2 ,....,y N . The sensor observations have known conditional 
distributions 



Detection Criterion 

The function of the detection process is to make a decision, U Q , about which 
state of the phenomena is true. The optimality criterion is a function 



that assigns to the event of deciding u- when H: is true a real number C-, i,j = 0,l, 
called the detection cost, so 



The objective of the decision rule will be to minimize the expected decision cost 



p(yi-y 2 ’->yN /H o)’ p(yi.y 2 --y N / H i )• 



( 1 . 2 ) 



J: U Q xH 



(1.3) 




(1.4) 



min E{J(u,H)}. 



( 1 . 5 ) 



An important ratio in our analysis is the constant given by 




( 1 . 6 ) 



Van Trees [6] showed that the average decision cost is given by, 




(1.7) 
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where P f and P d are the probability of false alarm 1 and probability of detection 2 
respectively. At this point we will make the assumptions that 

Cqi > C^j , (1*8) 



and 



Cjo > ^"oo * 0*9) 

These assumptions implies that it is more costly to err than to make a correct decision. 
Equation (1.7) can then be written in the form: 






p 4. r p 

on r o_~oi l 



< c ol * C 11 ) 



+ 



Pp (Ciq *Cnn ) 



oo 



P 1 < C 01 ' C ll ) 



Pf- p d 



P 1 < C 01 ' C ll 



)• 



( 1 . 10 ) 



Ignoring positive constants that will not affect our analysis, the average decision cost 
R is given by 



R = l + CP f P d . (1.11) 

1. The Centralized Detection (CD) Problem 

The problem of centralized binary hypothesis testing can be posed in its most 
general form as follows. For the structure of Figure 1.1 it is assumed that all sensor 
observations can be sent to one (central) location for processing. The function of the 
processor is to map the vector Y= [ y. y 2 ... y N ] l into the decision space U&subo(0,l) 

U Q : l - (0,1) (1.12) 



as follows; 



{ 0, H 0 is declared to have been detected 
1, Hj is declared to have been detected. 



Probability of deciding U =0 while Hj is true 
Probability of deciding U Q = 1 while Hj is true 
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Figure 1.1 Centralized Detection. 



Solution of the CD Problem 

The solution to the CD problem is 16] 

a) deterministic, so that the decision rule is a function of the observations 



Y : Yj xY 2 x...xY n (0,1), 



b) a likelihood ratio test, 



u 0 (yi.y 2 ’ 




if A (y 1 ,y 2 --,y N ) ^ t 

if A (Vj ,y 2 y N ) < t 



(1.14) 



(1.15) 



where 



A (y lf y 2 y N ) = 



flyi.y 2 -..y N /H t ) 

flypy2.--.yN/H0) 



c) and the threshold t is given by 



t = C . 



(1.16) 



(1.17) 



2. The Decentralized Detection (DD) Problem with Fusion 

Consider the structure of Figure 1.2 with H and Y being as before; the 
decisions L'j ,U, ,... and U N are sent to a fusion center. The activity of the fusion 
center is to make the global decision U according to some preset fusion rule. 

U 0 : U, xU 2 ..xU N r -* (0,1). (1.18) 

In the DD problem with fusion it is required to design local decision rules Uj,U 7t ... 
and U N , and a global fusion rule (1.18) so as to minimize the expected cost E{J(U o ,H)} 
incurred by deciding U Q = i when IT is true. 

Choosing an AND fusion rule apriori, Tenney and Sandell solved this problem 
for N=2. They set the decision rule as U = Lq U 2 and optimized the local decision 
rules. 
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PHENOMENA 





uO 



Figure 1.2 Decentralized Detection with Fusion. 



Solution of DD Problem with Fusion 

The solution to the DD problem with fusion is 

1. deterministic 



Yi : Y, -» (0,1) 
and 



Y 2 : Y 2 - (0,1) 



2. a likelihood ratio test for each detector 




where 



tfVyo* l i 

^ A i (Yi)< r i 




fly; /H 1 ) 

/H 0 ) 



3. with coupled thresholds tj and t 2 given by 
t _ c Pr(F 2 /yi) 

1 Pr(D 2 'y\ ) 



and 



, . c PrtF, /y,) 

2 Pr(D, /y 2 ) 

where Pr(Fj / y. ) and Pr(D ; /y. ) are respectively the conditional 
false alarm and the conditional probability of detection of the i^ 
the detector's observation. 



(1.19) 



( 1 . 20 ) 



( 1 . 21 ) 



( 1 . 22 ) 



(1.23) 



(1.24) 

probability of 
detector aiven 

W 



22 



Equations (1.23 ) and (1.24 ) are two coupled functional equations in ^ and t 2 . For 
general distributions, a functional expression for each of them in terms of its own 
observation and the other detector's decision is impossible. We shall consider the 
complexity of these decision rules later. A special case of the DD problem is the case 
of conditionally independent sensor observations, i.e. 

ity /y 2 ,H)= fly, /H) (1.25) 

and 



fly 2 /yi ,H)=f(y 2 /H). 



(1.26) 



In this case, the conditional probabilities in (1.23 ) and (1.24) reduce to 



t 



1 




(1.27) 



and 



t 



2 




(1.28) 



Equations (1.27) and (1.28) are two coupled algebraic equations in the form of 



ti=g 1 (t 2 ) and t 2 = g 2 ( t i) (1-29) 

since P fi and P di depend on t. . This coupling represents cooperation between the two 

sensors. The threshold equations are necessary conditions for optimality. There may 

be several local minima; each must be checked to assure the global minima. The 

threshold equations are strongly coupled for general cost assumptions. 

Tenney and Sandell came to the following conclusions: 

1. Increasing the signal-to-noise ratio improves the performance of the system. 
However a centralized system makes more efficient use of the increased 
information. 
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2. As the imbalance between the two detectors increases the performance 
improves. If the signal-to-noise ratio of one of the detectors goes to zero then 
the system decision is that of the other detector. This is equivalent to the 
performance of a CD system of the same signal-to-noise ratio. 

The case of conditionally independent observations has been considered by 
many authors. Sarma and Rao [7] extended Tenney and Sandell's results to the case of 
three sensors. They assumed a majority logic fusion rule and evaluated the threshold 
settings for some specific cases. Chair and Varshney [8] considered the problem of 
optimal fusion of N local decisions from prespecified local decision rules. Their 
optimum fusion structure is a weighted sum of local decisions according to their 
reliabilities. Reibman and Nolte [9] optimize both local decision rules and the fusion 
rule under the assumption of identical local decision rules. The global decision is then k 
out of N. They optimize the local decision rule for each k ,k= 1,2,.. .,N, then pick the 
value of k corresponding to the minimum decision cost. 

A sub-class of the DD problem with fusion, that will be referrd to as the 
"Second Opinion" problem, is the fusion of one's observation with another's decisions. 
An example of this is the second opinion in a medical examination, or even asking for 
legal advise. Ekchian [10] and Ekchian and Tenney [11] consider some specific 
topologies of this problem. Each decision maker has to make his decision based on his 
own observation and a predecessor's decisions. All the decision rules are likelihood 
ratio tests using the actual data. The thresholds are determined by incoming 
communication messages. The number of thresholds at each decision maker grows 
exponentially with the number of message inputs. Their results suggest putting the 
noisy sensor "up stream" in the detection network. 

Papastavrou and Athans [12] also consider the second opinion problem.They 
examine the structure of a primary decision maker, PDM, and a secondary decision 
maker, SDM ( a consultant ). The PDM makes his decision based on his own 
observation if it is of good quality. If his observation is noisy, the PDM asks, at a 
communication cost, the opinion of the SDM. Being activated by the request of the 
PDM, the SDM sends his decision to the PDM or ignores the request if his 
observation is noisy. In either case the PDM has to make a final decision. Again the 
thresholds are coupled. The threshold of the PDM is determined by the message of the 
SDM. 

This thesis is motivated mainly by three of the above works namely; 

1. Bayesian formulation of the DD problem by Tenney and Sandell [5]. 
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2. Extension of the DD problem to the Distributed Detection Networks by 
Ekchian. [10] 

3. Extention of the DD problem to the case of correlated sensor observations by 
Lauer and Sandell [4]. 

C. THE COMPLEXITY OF THE DD PROBLEM 

We saw that the DD problem can be solved optimally for conditionally 
independent sensor observations. If this condition does not hold local decisions are not 
likelihood ratio tests with constant thresholds. Tenney and Sandell show that for 
conditionally dependent observations, local decision rules are likelihood ratio tests but 
with data dependent thresholds (see e.g. (1.23 ) and (1.24 )). These two equations are 
coupled. This means that the observation of one sensor is necessary for the other 
sensor's decision, which contradicts the principle of decentralization. In terms of the 
terminology of the Theory of Combinatorial Complexity [13], Tsitisiklis and Athans 
[14] show that 

1. The DD problem with independent observations is a polynomial time problem. 

2. The DD problem with dependent sensor observations in its simplest form is a 
nondeterministic polynomial NP-complete. This means that exhaustive 
enumeration is necessary to find the optimum local decision rules. Optimality 
may be an illusive goal. So, suboptimal solutions must be sought. 

A suboptimal solution to the problem for the case of AND fusion was considered 
by Lauer and Sandell [4]. They considered the case of known signals in correlated 
noise. They took as a suboptimal solution local decision rules which are likelihood 
tests but having constant, not data dependent, thresholds satisfying the necessary 
condition of optimality. These thresholds are given by the implicit equations: 



At (Tj )=* 



c Pr(F 2 ,'T, ) 
Pr(D 2 ,'Tj ) 



(1.30) 



and 



A 2 (T 2 ) = 



c PKF, /T 2 ) 
Pr(Dj/T 2 ) ‘ 



(1.31) 
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D. CONTRIBUTIONS OF THIS THESIS 

We have reviewed the complexity of the DD problem and its current status. The 
research reported here has significantly advanced this status in several important ways. 
Specifically the contributions of this thesis have been to : 

1. Answer the question of the optimum fusion rule at the fusion center for the 
case of two sensors. 

2 . Specify the exact relation between the performance of the optimum fusion rule 
and the correlation coefficient between sensor observations. 

3. Solve the the second opinion decision problem. 

4. Solve the multi-level DD problem with fusion; i.e. detection with quantized 
sensor data for the known signal in noise case. 

5. Introduce the minimum risk quantizer. 

6. Grade the road between DD detection and CD detection. 

7. Optimally design quantizers for minimum mean square estimation. 

8. Present an efficient procedure to calculate parameters of a large variety of 
quantizers. 

E. ORGANIZATION OF THE THESIS 

The thesis is organized as follows. In Chapter II we consider the problem of 
fusion in DD. Optimum detection with quantized sensor data is considered in Chapter 
III, where the Quantized Detection algorithm, QD, is introduced. Numerical examples 
to illustrate the algorithm are given in Chapter IV. Generalization to the case of vector 
observations is presented in Chapter V. Optimum regeneration of sensor observations 
from their quantized versions and another sensor observation is considered in Chapter 
VI. A summary of the thesis, conclusions and suggestions for future research are given 
in Chapter VII. Proofs to some equations and FORTRAN programs to calculate 
parameters of the minimum risk and the minimum distortion quantizers are given in 
the appendices. 
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II. OPTIMUM FUSION OF LOCAL DECISIONS 



In this chapter the important question of the optimum fusion rule will be 
answered. The relationship of the optimum fusion policy to the ratio of costs and the 
correlation coefficient between observations is determined. 

A. INTRODUCTION 

Distributed Detection with fusion is a two level optimization problem. The 
problem can be formulated in the following three ways: 

1. Local Decision Optimization 

The first way is to select the fusion rule apriori and optimize the local decision 
rules accordingly. Setting the activity of the fusion center as AND fusion, Tenney and 
Sandell [5] derived optimum local decision rules for a pair of spatially separated 
detectors with conditionally independent observations. They prove that local decision 
rules are simple likelihood ratio tests with constant thresholds. The thresholds are the 
solution of a pair of coupled algebraic equations that correspond to the global 
minimum of the detection cost function. They also show that for the case of correlated 
observations local decision rules are likelihood ratio tests but with data dependent 
thresholds. Functional solution of the threshold equations in the later case violates the 
principle of decentralization. Realizing the difficulty of the problem in the case of 
correlated observations, Lauer and Sandell [4] designed suboptimal local decisions for 
AND fusion. Their local decision rules are likelihood ratio tests with constant 
thresholds satisfying the necessary conditions of optimality. Kovatana [15] considered 
AND fusion for two detectors. Fefjar [16] compared AND to OR fusion for two 
detectors. He claimed that OR is better than AND. Stearns [17] contradicts Fefjar's 
results. He showed by an example that OR combining is better for higher cost of 
missing the target while AND combining is better for higher cost of false alarms. 

2. Fusion Rule Optimization 

In the second formulation of the problem, local decision rules are set apriori. 
Optimization is carried out with respect to the fusion rules. An example of this 
situation could be factory built sensors that cannot be adjusted. Assuming local 
threshold settings Chair and Varshney [8] prove that for the case of conditionally 
independent sensor observations, the optimum fusion rule is a likelihood ratio test that 
sums local decisions weighted according to their reliability. 
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3. Global optimization of the Local decisions and the Fusion Rule 

The third formulation involves optimization at both levels. Here local 
decisions are optimized for every possible fusion rule. The optimum fusion rule is the 
one that minimizes cost. 

The main issue of this chapter is the global optimization of the DD system for 
general correlated observations. First we will state the main results for the case of N 
conditionally independent and identically distributed sensor observations. Then, the 
. problem of fusing two local decisions of sensors with correlated observations is 
considered. 

B. GLOBAL OPTIMIZATION OF DISTRIBUTED DETECTION 

In CD all sensor observations are available at one central processor for detection. 
The decision rule in CD is a likelihood ratio test in the observations y 1( y 2 ,...,y N . It 
declares Hj is true if the likelihood ratio 

A (y r y 2 ,...,y N ) ^ c , (2.1) 

otherwise it will declare H Q to be true. 

In DD only local decisions are sent to the central processor ( fusion center). The 
objective of the fusion center is to mix ( fuse ) the local decisions into a single global 
decision with minimum decision cost. So given the local decisions the observation 
space of the fusion center consists of 2^ discrete points. The activity of the fusion 
center is to divide this space into two decision regions Z Q and Zj. The decision rule of 
the fusion center is a likelihood ratio test [S.] The fusion center declares Hj is true if 

A (u 1 ,u 2 ,...,u N ) > C. (2.2) 

otherwise it will declare that Hq is true. In the special case of conditionally 
independent and identically distributed observations, the fusion rule is a k out of N 
rule. Reibman and Nolte [9] considered this problem. Assuming the same decision rule 
for every detector they optimize local decisions for every k, k= 1,2,...,N then pick the k 
with the minimum decision cost. 

If sensor observations are not conditionally independent, there is no guarantee 
that local decisions are simple likelihood ratio tests. The problem turns out to be NP- 
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complete which needs exhaustive enumerations to find the optimum decision rules [14.] 
Moreover if sensor observations are not identically distributed, there are as many as 

,N 

2 Z possible fusion rules for the N sensor decisions. Any algorithm that goes through 
the entire fusion list optimizing local decisions will be impractical 3 for N ^ 6. 

Our approach to avoid this exhaustive enumeration is the following: 

1. We assume that local decisions are likelihood ratio tests with constant 
thresholds. Again we emphasize that this assumption is valid only for 
conditionally independent observations, there is no guarantee that it is correct 
for correlated observations [5], So the constant threshold likelihood ratio test is 
optimum for conditionally independent observations and perhaps suboptimum 
for correlated observations. However the solution tends to the optimum 
solution as the correlation coefficient tends to zero [4]. 

2. Those fusion rules which agree with the CD solution will be tested. The rest of 
the fusion rules will be disregarded. The meaning of this will be made clear in 
the following example. 

Let us consider the case of two sensors (N =2 ) in detail. To be explicit, 
consider detection of known signals in gaussian noise. The sensor observations are 
given by: 



H„ : yj = Dj . i= 1,2 (2.3) 

and 

Hj : y. - a. + n ; , i- 1,2. (2.4) 

The a ; 's are positive constants and N = [n 1 n 2 f is vector of zero mean with 
covariance 



K - C 3 (2 ' 5) 

where p is given by 

P = E { n 2 n 2 }. (2.6) 



3 A computer that spends 1 p second in every optimization process, will spend 
40000 years to determine the optimum fusion rule, for N = 6. 
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The threshold equation of the CD problem is given by [6], 



( a l -P a 2 )7i + ( a 2 * P a i )y 2 = ( a i 2 +a 2 2 ' 2 P a i a 2 )/ 2 + ( 1 -P 2 )log(Q (2.7) 

which is a straight line in the yj y 2 plane. Figure 2.1 shows decision rules based only 
on Dj, only on D 2 , both decision rules together, and the decision rule of CD. 

The global optimization requires optimizing local decision rules for every fusion 
rule then picking the fusion rule with minimum average cost. The observation space of 
the fusion center consists of four discrete points (0,0), (0,1), (1,0), (1,1). Any fusion 
rule divides this space into two decision regions Z j and Z Q . There are 2 4 =16 methods 
to subdivide four points into two groups. Table 1 contains a list of those fusion rules. 
Some special cases for the detection problem are as follows. 

1. If Cjq -* go, i.e. the cost of missing the target is extremely high, the CD 
solution assigns all the observation space to Z Q . The fusion center can perform 
the same. This is fusion rule one. 

2. Similarly if C Qj -> °o, the fusion center will always decide Hj, this is fusion rule 
two. 

3. If a 2 = p a 1? the CD will decide based only on y r So will the fusion center. 
This is fusion rule three. This can only happen when aj ^ a 2 . 

4. If aj = p a-,, the CD will decide based on y,. This is fusion rule four. This can 
only happen when a, £ a r 

The first two situations represent extreme conditions of C. The next two conditions 
deal with specific values of p. We also distinguish the following two cases. 

Case a 

-1 ^ p ^ min( aj ,a 2 )/max( a t ,a 2 ). 

In this case the Vj and y 2 intersections of the threshold equation (2.7 ) are of the same 
sign. 

Case b 

min( a t ,a 2 )/max( a t ,a 2 ) < p < 1. 

In this case the y t and y 2 intersections of the threshold equation are of different signs. 
We shall consider these intervals of p when we study the effect of correlation between 
sensor observations. 

The CD threshold m the y 1 y 2 plane suggests assigning the decision point (0,0) to 
Zq and (1,1) to Zj. The fusion rules from 5 to 14 do not do this. They either assign 
(0,0) to Z 1 or assign (1,1) to Z Q or assign (0,0) and (1,1) to the same decision region. 
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Figure 2.1 Decision Rules. 
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TABLE 1 

EXHAUSTIVE FUSION LIST OF TWO DECISIONS 



Rules 


Zo 


Z, 


Comments 


1 


<t> 


(O.O).(0, 1 ).( 1 ,0),( 1 . 1) 


C 01 “* 33 


2 


(0,0),(1.0),(0,l),(l,I) 


d> 


C.o-* 00 


3 


(0.0),(1,0) 


(O.I).(l.l) 


u 0 -l ; 2 


4 


(0,0), (0.1) 


(l.O).(l.l) 


-L : , 


5 


(0.1) 


(0.0).( l.O).(l.l) 




6 


(1.0) 


(0.0 1.(0. t).(l.l) 




; 


(1.1) 


(0.0).( I.O).(O.l) 




! s 

\ 


(0.1).(1.0) 


(0,o).( l.l) 




9 


(o.iuu) 


1 

(DU).(l.O) 




10 


(1.0 >.(l.l) 


(O.O).(iM) 




11 


(0.0)41.1) 


(o.n.(i.o) 




r 

12 


(o.n.(i.oi.(i.i) 


(O.U) 




13 


(O.O).(U. 1 ).( 1,1) 


(1.0) 




14 


(<».<>>.( I.u>.(l.l) 


( f >.l) 




15 


(o,i>).((>.n.(i.o> 


(1.1) 


AND 


16 


(0.01 I 


(0.1 ).( 1 .(»).( 1.1 ) 


OR 
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We will not consider these ten fusion rules further. The remaining two decision rules 
are the AND fusion and the OR fusion. Let us now consider optimizing each of 
them. 

1. ' AND ' Fusion 

In AND fusion u Q is given by: 

u 0 = Uj u 2 . (2.8) 

The individual rules are given by assigning y ; to Zj if 

7i > Tj ,i= 1,2. (2.9) 

Otherwise they assign it to Z Q . 

The probability of detection P d (AND) and probability of false alarm PLAN'D) 
of the fusion center are given by: 



oo oo 

P d (AND)=J L Ry^yj/H, ) dy, dy 2 (2.10) 

! 1 *2 

and 

oo oo 

Pf (AND)= f J fty,,y 2 /H 0 ) dy, dy 2 (2.11) 

! 1 l 2 

It has been shown in Chapter I that, to within positive multiplicative and 
additive constants, the average decision cost is given by 

R = 1 + C P f - P d . (2.12) 

Substituting for P d and P f in (2.12) from (2.10) and (2.11) expresses R(AND) as a 
function of Tj and T 2 . The necessary conditions for optimality are 

d R Id Tj = 0 and d R/3 T 2 = 0 , (2.13) 

which can be written in the forms: 
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*T lf y 2 / Hj ) dy 2 



(2.14) 



,oo cp 

c J T f(T,,y 2 /H„) dy 2 = £ 



and 



°o ' co 

C r Ky,.T 2 /H 0 ) dy, - f KypTj / H, ) dy, 
1 1 1 1 



(2.15) 



Applying Bayes rule and rearranging terms, one can write (2.14) and (2.15) as follows: 



oo 



J T ny 2 / T i > h q) d >2 



Aj (Tj ) = C 



J T fly./Tj ,Hj) dy 2 

l 2 



(2.16) 



and 



oo 



t T Hn/T, ,H 0 ) dy, 



A, (T,)- C 



CO 



L fty/Tj .Hj) d Xl 

‘l 



(2.17) 



To insure minima the Hessian matrix of R with respect to Tj and T-, 



(2.18) 



'd 2 R Id T 2 d 2 R Id Tj d T 2 “j 

.< d 2 R id T 2 d Tj d 2 R,'d T 2 2 J 



(2.19) 



must be positive definite. Optimum threshold settings Tj and T-, are the solution of 
(2.16) and (2.17) that corresponds to the global minima, so all possible solutions of 
(2.16) and (2.17) must be tried. The coupling between (2.16) and (2.17) to determine 
the thresholds represents the cooperation that can occur between the two local 
detectors to minimize the overall decision cost. 



34 



2. 'OR' Fusion 

The decision of the OR fusion is given by 

u Q = Uj + - Uj u 2> (2.20) 

The probability of detection P d (OR) and probability of false alarm P f (OR) are given 
by 



P d (OR)- 1 - f 



1 

00 



.00 



^y v y 2 / H i ) d yi d y2 



and 



P f (0R)= 1 - 





/Hq ) dyj dy 2 



( 2 . 21 ) 



( 2 . 22 ) 



while the necessary conditions for optimality are 

C 2 W T i’ H o> d y2 

A (T, ) - C — (2.23) 

SJ flj/T, ,H,) dy 2 

and 

£' «y/r 2 ,h 0 ) dy, 

A (Tj) - C — . (2.24) 

U ^i / t 2 .H.) dy, 

Again the Hessian matrix must be positive definite. 

3. Solution of the Nonlinear Threshold Equations 

The pair of coupled equations (2.16), (2.17) for the AND fusion and (2.23) 
and (2.24) for the OR fusion can be solved using Max's technique [18], The technique 
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is summarized as follow: pick a value of Tj and calculate T 2 from (2.16) or (2.23 ). If 
the calculated value of T 2 does not agree with that value calculated from (2.17) or 
(2.24) then Tj must be chosen again. This approach is time consuming. Another 
approach is the method of successive substitution [19]. We fust put the two equations 
in the form 



then start with a reasonable guess for (Tj ) 0 and (T 2 ) Q . A suitable initial guess is the 
locally optimum solutions, i.e. the thresholds that would optimize the detection if each 
sensor works alone. These will be denoted by T llo and T-, lo . For known signals in 
gaussian noise these are 



4. Numerical Results 

We have solved the threshold equations for both fusion rules for a t = 1.7 and 
a-, = 2.3 for several values of p and C. 

To compare AND and OR fusion, define K as the ratio of the AND cost to 
the OR cost. 



We have also computed the Receiver Operating Characteristic 4 (ROC) curves of 
classical communication theory [20] for each fusion rule. 

Figure 2.2 shows the ratio K as a function of C for p = 0, 0.2 , 0.4. The 
figure shows that AND fusion is optimum for C £ 1 and OR fusion is optimum for 
lower values of C. The same is clear from Figure 2.3; ROC curves of AND fusion are 
above those of OR fusion for C ^ 1 and lower otherwise. The performance difference 
becomes smaller as the correlation coefficient increases. Also the figures show that the 
performance degrades for both fusion rules as p tends to one. This is in sharp contrast 
to CD which has perfect detection for p= 1. 

4 P d as a function of P f 




(2.25) 



< T i )o = V 2 + iog(Q/a i • 



(2.26) 



1 + C P f (AND) - P d (AND) 
1 + C P f (OR) - P d (OR) 



(2.27) 
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Figure 2.2 Ratio of Costs of AND and OR Fusion Rules 
a^ ~ l*7j a 2 = 2.3. 
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Figure 2.3 ROC Curves of AND and OR Fusion Rules 

= 1.7, a 2 = 2.3. 
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The same effects can be concluded from Figure 2.4 and Figure 2.5. Figure 2.4 
shows the ratio K as a function of C for a 1 = 1 and a 2 = 2 and for p = 0, 0.25, 0.5. 
Figure 2.5 shows the ROC curves for both fusion rules for the same case. The figure 
shows that AND fusion is optimum for C ^ 1 and OR fusion is optimum for lower 
values of C. 

C. THE EFFECT OF CORRELATION BETWEEN SENSOR OBSERVATIONS 

So far we have answered the question of the optimum fusion rule. For C ^ 1 
AND fusion is optimum. Let us now examine the effect of the correlation coefficient p 
on the performance of AND fusion for C ^ I (its range of superiority). We assume 
without loss of generality that a 2 is greater than a r The two necessary conditions for 
optimality of AND fusion are (2.16) and (2.17). For the problem of known signal in 
gaussian noise these can be written as: 



erfc 



A; (T, ) - c 



erfc 



f T - ^ T I \ 

\ V (1-P 2 ) / 

/ T 2 ~ a 2 ~P ( T 1 ’ a i ) 

1 V (i-p 2 ) J 



(2.28) 



and 



erfc 



A 2 (T 2 ) 



= c 



fVPjil 

Wd-P 2 ) / 



/ T] -a! -p (T 2 -a, ) ^ 

1 V (i-p 2 ) j 



(2.29) 



erfc 



Notice that C appears only as a multiplicative constant in the two equations. The role 
of p is not that obvious. Examining the two equations leads to the following insights 
about the role of p: 

1. Tj =-oo and T 2 = T 2[ is a solution. This corresponds to the decision rule of 

d 2 . 

2. T 2 = -co and T l T llo is a solution. This corresponds to the decision rule of 



D 



r 



3. If a 2 is greater than a t we expect the performance of D 2 alone to be better than 
that of Dj alone and that of the selfish decision rule in which each detector tries 



to minimize its own detection cost, not the system decision cost, by using T 



lto' 



21o‘ 
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Figure 2.4 Ratio of Costs of AND and OR Fusion Rules 

a t = 1, a 2 =2. 
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T 



T 



T 




Figure 2.5 ROC for AND and OR Fusion Rules 
a t = l,a 2 = 2. 
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We now prove three lemmas concerning these equations. 
1. Lemma 1. 

For p ^ a x /a 2 , 



and 



T < T 
l l ~ l llo 



T < T 

l 2 “ l 21o 



where T a is the optimum threshold of the ith detector operating alone. 

Proof: 

Since the argument of the complement of the error function in each 
denominator is less than the argument in the corresponding numerator, the 
fraction is always less than one. This implies that 

Aj(Tj) < C ,i= 1,2. 

2. Lemma 2 

For p =aj /a 2 , the only possible solution of (2.28) and (2.29) is: 



and 



T = T 

1 2 1 2lo 



T, =-30. 



Proof: 

For p = a t / a 2 equation (2.29) becomes 



A 2 (T 2 ) = C =a 2 (t 21o ) 



(2.30) 



The corresponding value ofTj is Tj = -oo. 

3. Lemma 3 

For p ^ a t /a, the optimum solution for T^ and T 2 is: 
and 



T^-co 



T = T 

1 2 1 21o‘ 

This means that the decision of the optimum AND fusion is that of D 2 . 

Proof: 

Recall that the CD threshold line divides the observation space into two 
decision regions. For positive signals the following inequality is satisfied in the 
region to the right of the CD line: 



Cfty ,y 2 /H 0 )< fl yi ,y 2 /Hj ) 



(2.31) 
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The converse of this inequality is true in the left region. The decision region Zj 
of any other decision rule contains areas from the right and from the left of the 
CD line. Areas to the right will have a negative contribution to the decision 
cost while areas to the left will have positive contributions. Now assume that T l 
and T 2 , where both are finite, satisfy the necessary condition (2.28) and (2.29). 
We shall prove that they cannot correspond to the optimum solution. The 
finite point ( Tj ,T 2 ) either lies to the left or to the right of the CD threshold 
line as shown in Figure 2.6 a and b respectively. In Figure 2.6 a the intersection 
of the CD line with the line y 2 = Tj is a better solution since it excludes an 
area in which C ffy ,y 2 /H 0 ) is greater than ffy , y 2 /Hj ). A better solution 
than this has the same T 2 but with Tj = -co since the added area has negative 
contribution to the cost. In Figure 2.6 b, T 1 =-co and T' 2 is a better solution 
than Tj and T 2 , since the added area has a negative contribution to the cost. 
In both cases Tj = -co is the optimum solution and the corresponding optimum 
value of T 2 is T, Io . 

As a result of the above three lemmas it is clear that 

1. Any solution of the necessary conditions must satisfy 



2. The performance of the AND fusion saturates to that of D, alone for p ^ 
a l /a 2> We might recall that the threshold line of the CD system changes slope 
at that value of p. We w T ill refer to this value of p by p cr . This result is in 
contradiction w'ith Lauer and Sandell's results [4] which shows performance 
continuing to degrade with increasing p for 



Limiting behavior for = p-l. 

For p =-l the joint probability density function f(y A ,y 2 /H 0 ) has values only on the 
line Vj = -y r So any threshold values Tj and T-, such that Tj =-T 2 will produce 
AND fusion with zero probability of false alarm. This can be visualized from Figure 
2.7 . Consequently, P d will be given by 






(2.32) 



Maximizing P d with respect to T 2 yields 



T 2 =( a 2 * a i )/ 2 • 



(2.33) 



For the special case of equal SNR sensors, T-, = 0. 
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Figure 2.6 Optimum Tj and T 2 for p S a t / a 2 , 
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Figure 2.7 Centralized Threshold Line For Different 
Values of p. 
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D. NUMERICAL RESULTS 

The average decision costs vs p for a 1 = a 2 = 2, and C= I are shown in Figure 2.8. 
Threshold values Tj and T 2 vs p for the same case are shown in Figure 2.9. Figures 
2.10 and 2.11 show the same for C= 10. 

These four figures for the case of equal signal-to-noise ratio show that the two 
detectors cooperate with each other using the same decision rule ( equal thresholds ). 
Their threshold is an increasing function of p. The limit of this threshold as p — ► -1 is 
zero. This behavior agrees with (2.33). The limit of the threshold as p -* 1 is Tj . This 
is because for p -*■ 1 the two systems have identical observations. 

The detection cost curves show that the cost is an increasing function of p. The 
curve of the AND fusion has the same shape as the curve of the CD system. Both 
systems attain their best performance at p = -l. They have the same worst performance 
for p= 1. 

Figures 2.12 and 2.13 represent the case of unequal SNR sensors for C=l. 
Figures 2. 14 and 2. 15 show the same for C= 10. 
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Figure 2.8 Average Decision Costs for Equal SNRs,C= 1. 
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Figure 2.9 Threshold Value for Equal SNRs,C= 1. 
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Figure 2.10 Average Decision Costs for Equal SNRs,C= 10. 
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Figure 2.11 Threshold Value for Equal SNRs,C= 10. 
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Figure 2.12 Average Decision Costs for Unequal SNRs,C= 1. 
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Figure 2.13 Threshold Value for Unequal SNRs,C= 1. 
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Figure 2.14 Average Decision Costs for Unequal SNRs,C= 10. 




Figure 2.15 Threshold Value for Unequal SNRs,C= 10. 
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These four figures for the case of unbalanced sensors show that, the two 
detectors cooperate using different thresholds. The threshold of the higher signal-to- 
noise detector is an increasing function of p while the other threshold is a decreasing 
function of p. 

The cost curves show that the fusion rule has its best performance at p =-l. 
Both DD and CD have their worst performance at p= p CT . For p > p CT the 
performance of the optimum fusion rule is the same as the detector of the higher 
signal-to-noise ratio. Recall that CD system has perfect detection for p = 1 when the 
SNR's are unequal. As C increases the average cost of each system increases. This can 
be explained from the expression for R in which the probability of false alarm is 
weighted by C. 

E. DISCUSSION AND CONCLUSIONS 

We have shown that the optimum fusion rule is determined by the ratio of costs 
and the apriori probabilities. For equal error costs AND and OR fusion rules are 
equivalent. This is not surprising since each system turns out to be the minimum 
probability of error detector; thresholds are adjusted such that 1 -P d = Pp It might also 
be noted that the optimality of the fusion rule is independent of the correlation 
coefficient and the signal-to-noise ratio in this case. We also note that the detection 
cost of the optimum fusion rule has its minimum value at p = -l. It has its maximum 
value at p = a 1 /a 2 . The performance saturates at the cost of decision of the detector of 
higher SNR. In the interval (p e [a 1 /a 2 ,l]), the optimum fusion rule ignores the 
decision of the detector of lower SNR. As a good dynamical example that agrees with 
this result is the switched diversity combiner [21] in fading environments and its 
centralized counterpart, the maximum ratio diversity combiner [22]. Recall that for 
unequal SNRs the performance of the CD system improves in this interval and has 
perfect detection for p =1. Also it is important to note that the optimum thresholds 
of the individual observers are not the same as if they were operating independently, 
but must be determined by simultaneous solution of two coupled nonlinear equations. 
This represents the cooperation between the two detectors to work as a team. Lastly 
the performance difference between CD and DD is due to the information loss in local 
data processing. However DD has fewer requirements on the communication channel 
in contrast to CD which requires infinite bandwidth. A compromise between these two 
extremes is to allow more information than just decisions to be sent to the fusion 
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center. This is the concept behind the Quantized Detection algorithm considered in the 
following two chapters. 
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III. DETECTION USING QUANTIZED SENSOR OBSERVATIONS 



A. INTRODUCTION 

So far detection with sensor observations has been described using two methods. 
In the first method all sensor observations are sent to some central processor which 
makes a decision based on a likelihood ratio test. In the second method only local 
decisions are sent to the central processor which fuses these decisions into a global 
decision. While the first method is very easy to design it requires in principle infinite 
bandwidth communication channels. The second method requires only one information 
bit per detection. Detection with quantized sensor observations will be introduced in 
this chapter. The main goal of the chapter is to grade the road from the DD problem 
to the CD problem. It will be referred to by Quantized Detection, QD. The 
performance improvement of the DD problem will be traced as the amount of 
information delivered to the fusion center increases. 

First let us consider the problem of the Primary Decision Maker (PDM) and its 
quantized second opinion (consultant). We will prove three theorems concerning the 
decision rule of the PDM. Then fusion of two quantized observations of an arbitrary 
number of levels will be considered. As a special case, fusion of two sensor 
observations, one quantized to N levels and the other to N +' 1 levels, will be proven 
equivalent to the PDM and an N-level quantizer. Comparison between different 
configurations will follow. 

B. TEAM DECISION OF A PRIMARY DECISION MAKER AND A SECOND 

OPINION QUANTIZER. 

1. Formulation of the PDM Problem 

Consider the structure of Figure 3.1 in which y^ is quantized into yj by the 
quantization rule a of N levels. 

a:Y,-.Yi q . (3.1) 

The primary decision maker will make his decision u o , about the phenomena H based 
on its own observation y 2 and the quantized observation yj . 
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Figure 3.1 Configuration A. The Primary Decision 
Maker and its Quantized Consultant. 
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The problem of the PDM is 

1. to design the quantization rule a i.e. to specify the set of N points 

-oo = X T < X 2 < ... < X N < oo 
that defines the quantizer intervals, and 

2. to design the decision rule y 2 



y 2 -Y lq xY 2 ->(0,\) 



(3.2) 



in order to minimize the decision cost. 

2. Problem Analysis 

Our approach is as follows. We first design the optimum Bayes decision rule 
given a set of quantizer parameters. Next, the average cost is expressed as a function of 
these parameters. We then minimize the average cost with respect to them. 
a. The Optimum PDM Given Some Quantization Rule a 

We have shown in Chapter I that, to within an additive and a 
multiplicative positive constant the average cost is given by [6j 

R = C P f - P d (3.3) 

where C is the ratio of error costs and P f . and P d are the probability of false alarm and 
probability of detection respectively. The PDM receives a quantized level y^ = Qj . 
He will make his decision on the basis of his own observation v, and v. . The 
performance of the the primary decision maker, given some quantization rule a, is 
given by the following lemma. 

Lemma 3.1 

The probability of detection and probability of false alarm of the Primary Decision 
Maker are given by: 




7 fl>vy 2 / H i ) d Yi dy 2 

eZ U 



(3.4) 



and 
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(3.5) 




r X i + l 

Jx i 




Jlyi.y 2 / H o ) d yi d y 2 



where is the decision region Zj given that y 1 €[Xj,Xj + 1 ]. 

Proof: 

The proof is given in Appendix (A). 

The decision rule of the Primary Decision Maker is given by Theorem 3.1. 
Theorem 3. 1 

Given y lq and y 2 the decision rule of the Primary Decision Maker of Figure 3.1 is 

1. deterministic 



Y 2 : Y lq X Y 2 -* ( 0 - 1 ) 



(3.6) 



2. a likelihood ratio test 



U f I if A (y 2 ) ^ 0j(y 2 ) 

U ° i 0 if A (y 2 ) < 0.(y 2 ) 

where A (y 2 ) = f(y 2 /Hj )/f(y 2 /H 0 ) 

3. the threshold function ©j (y 2 ) is given by 



»j = 1,2,... ,N 



(3.7) 



J^ i+1 ny 1 /y 2 .H 0 )dy 1 

0j (y 2 ) = CJ ,j= 1,2,. ..N. (3.S) 

J x i+ ^i''y2’ H i) d yi 

J 

Proof 

We first insert (3.4) and (3.5) into (3.3). Each term of the detection cost (3.3) is then 
given by 



X. 



R i = L 7 J c fty -y 2 / H o ) - ^1 -y 2 / H i ) ] d yi d y 2 

y 2 ez -n A i 



(3.9) 



To make Rj in (3.9 ) negative an optimum decision rule assigns y 2 to Zj if 
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(3.10) 



c $ +i <yi /H o ) d yi • ft +1 <&i .y 2 / H i ) d yi 2 0 j- ia-n 

, A i 

otherwise it will assign y 2 to Z Q . 

Applying Bayes rule and rearranging terms, decision rule (3.10) can be 

written as 

t* i+1 «y I /y 2 ,H 0 )dy 1 

A(y 2 )*C-U . j= 1.2....N (3.11) 

S x i+1 f(y 1 /y 2 .Hi)dy 1 

J 

which completes the proof. 

b. Optimum Quantization of Y j 

According to Theorem 3.1, the decision rule of the PDM is a likelihood 
ratio test with data dependent threshold. The threshold depends on the choice of X.'s. 
To find an optimum solution for the X- 's is not any easier than that of the DD 
problem. Recall that for the DD problem optimum solutions are possible only for the 
case of conditionally independent observations. Only suboptimal solutions are possible 
for the case of correlated observations. We will not expect more for the QD problem. 
Let us consider each case separately. 

3. Conditionally Independent Observations 

Under the assumption of conditionally independent observations, i.e. 



) = flyj/ H ) (3.12) 

the decision rule of the Primary Decision Maker can be simplified. This decision rule is 
given by the following corollary of Theorem 3.1. 

Corollary 1 

Assuming conditionally independent sensor observations, and given y T and y 2 , the 
decision rule of the Primary Decision Maker of Figure 3.1 is 
1. deterministic 



: Y lq X Y 2 - ( 0 • 1 ) 



(3.13) 
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2. a likelihood ratio test 



u 



o 




(3.14) 



where A (y 2 )= ffy 2 /Hj )/f(y 2 /H 0 ) 
3. the threshold 0. is given by 





(3.15) 



Proof 



By applying condition (3.12) in the threshold equation (3.8 ) one obtains (3.15) which 
completes the proof. 

Let us denote the conditional probability of detection and the conditional 
probability of false alarm of the PDM given that the quantization level of Vj is 
received by P^. and P^. Let T. be the set of all points y 2 for which 



a 2 (y 2 ) a ©j 



(3.16) 



Then P .. and P r can be written as 

dj fj 




(3.17) 



and 



? n - f <P . /Ho ) dy 2 



(3.18) 



Equations (3.4) and (3.5) are now given by 
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(3.19) 



PdiJJ fx! +lf(y ‘ /Hi)d) '‘ M y 2 /H i )dy 2 

and 

p f = f, J^ + to, /Ho ) ay, J Jly 2 /h 0 ) dy 2 . (3.20) 

Substituting (3.19) and (3.20) in (3.3), then differentiating R with respect to Xj 
,{j = 2,3,...N} will yield necessary conditions for optimality of the set of N equations. 

C If ,y 2 /Hq )dy 2 -f nx k ,y 2 /H 0 )dy 2 ]- 

T k T k-1 

f J ^ X k ’^2 / H 1 > d >*2 -J[» RX k ,y 2 /H 1 )dy 2 ] = 0 ,k = 2,3,..,N. 

1 k *k-l 

Applying Bayes rule and rearranging terms, (3.21 ) can be written in the following w r ay. 



f T ^V H o) d ^2 • J , F ^2' H o) d ^2 

A (X. ) = C -7 — ? ,k = 2,3,...,N 

k J T r <>V H l) d >2 



(3.22) 



V F 



k-l 



The set of N-l necessary conditions (3.22) are general for any statistics of y 1 . For the 
special case w’hen A (y 2 ) is monotonic in y 2 , let T. be the value of y 2 for u’hich 



a -A 2 (T.) ,j= 1,2,...,N. 



(3.23) 



So T. is given by 



j* i+ V H o> d y, 

A (T. )- C — ^ 

Ix i+ V H i) d yi 



,j = 1,2,...N 



(3.24) 



(3.21) 
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For this case of monotonic A (y 2 ) the set of necessary conditions for optimality (3.22) 
can be written as 



fy f(y 2 / H o) d y 2 - J°^fl:y 2 /H 0 ) 

A (X. ) = C ^ ^ ,k=2,3,...,N (3.25) 

K 00 00 

J ^y 2 / H i) d y 2 - L fly 2 /H 1 )dy 2 

*k l k-l 

Equivalently we can write (3.25) in the form 



J ^ k -‘fTy 2 /H 0 ) dy 2 

A (X k ) - C -T ,k = 2,3 N . (3.26) 

dy 2 

l k 



P d and P f in this case are given by 




I ±1 oo 

f x ; + 'fty, /H, ) dy, J T Hy 2 /H, ) dy 



T. 



and 



P 



f 



= Y' 
i^l 



j^ i + 1 n y l /H 0 ) dVj J T fly, /H 0 ) dy. 



(3.27) 



(3.28) 



Equations (3.24) and (3.26) are only necessary conditions for optimality for 
monotonic likelihood ratio. They correspond to minima if the Hessian matrix 
[.3 2 R/£X.dX.] is positive definite. All solutions must be checked for the global minima. 

4. Solution of the Primary Decision Maker Problem with Independent Sensor 
Observations and Monotonic Likelihood Ratio 

The following theorem summarizes the above solution of the PDM with 
independent sensor observations and monotonic likelihood ratio. 
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Theorem 3.2 

The decision rule of the Primary Decision Maker with a Quantized Consultant (for 
independent sensor observations and monotonic likelihood ratio) is; 

1. deterministic 



: Yiq x Y 2 -* ( ° , 1 ) 



(3.29) 



2. a likelihood ratio test 



u 




if A (y 2 ) > 0. . 

if A (y 2 ) < Q ] ,] 



1,2,. .,N 



where A (y 2 ) = fl[y 2 /Hj )/fIy 2 /H 0 ) 

3. the threshold function © (y 2 ) is given by 



(3.30) 



0j = c ^ (3.31) 

J x i + I «y 1 /H 1 )dy 1 

J 

The optimum set of quantizer interval end points must satisfy the set (3.26 ), where 
T k 's are given by (3.24). All possible solutions must be checked for the global 
minimum cost. 

5. The Case of Correlated Observations 

We now move to a more realistic situation by removing the condition of 
independent sensor observations. In many radar and sonar problems noise in nearby 
sensors is likely to be correlated. As we mentioned before the decision rules (3.11) are 
likelihood ratio tests with data dependent thresholds. It is impossible to come with 
their optimum functional expressions [4.] A suboptimal solution for the case of 
correlated observations is to use likelihood ratio tests with constant thresholds as local 
decision rules. These constant thresholds for y 2 are the values of y, for which the 
inequality (3.11 ) is an equality, i.e.; 
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,k= 1,2,. ..,N . 



(3.32) 



.X 



j;‘ k+1 f(yi/T k .Ho)dy 1 



A (T„ ) - C 



X 

.X, 



J k+1 f(yi/T k ,Hi)dyi 
A k 



In terms of these thresholds T k 's and the quantizer points X k 's one can write 
expressions for the probability of detection and the probability of false alarm in the 
form of (3.4) and (3.5). Substituting for P d and P f in (3.3) and differentiating R with 
respect to X k for k=2,3,...N yields the following set of necessary conditions for the 
case of monotonic A 2 (y 2 ) : 



J^-'Ry 2 :X k ,H 0 ) dy 2 

A (X R ) = C ,k= 2,3,..., N. (3.33) 

f T k -‘(byx k ,H 1 ) dy 2 

‘k 



The set of equations in (3.32) and (3.33) constitute 2N-1 equations that specify the 
quantizer interval end points {X k } for Vj and the thresholds (T k ) for y 2 . 

C. TEAM DECISION OF TVVO QUANTIZERS AND A FUSION CENTER 

In this section we will consider the problem of making a global decision based on 
two quantized observations. 

1. Formulation of the QD problem 

For the structure of Figure 3.2, y t is quantized into N levels by the 
quantization rule cij 



a. 



Y, 



iq 



and y 2 is quantized into M levels by the quantization rule a 2 



(3.34) 



a. 



Y, -» Y 



2q 



(3-35) 



The quantized values y lq and y, q are sent to the fusion center w’hich must decide which 
state of the phenomena is true. It is required to design the quantization rules otj and 
a, and the decision rule y 
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(3.36) 



to minimize the global cost. 

2. Problem Analysis and the QD Algorithm 

The observation space of the fusion center contains NM points to be divided 



solution is to approximate the threshold equation of the corresponding CD problem by 
a piecewise curve in the y j y 2 plane. This is illustrated in Figure 3.3. 

The figure shows a schematic diagram of a CD threshold curve and its 
staircase approximation. The approximate curve consists of segments of straight lines 
connected together. The coordinates of the connecting points will play the role of the 
interval end points of the quantizers. Let us first w r rite an expression for P d and P f in 
terms of these point coordinates. If this expression of the cost is minimized with 
respect to each coordinate there will be as many equations as the number of 
coordinates. Solving these equations simultaneously yields the quantizer parameters. 
This is the core of the QD algorithm which is summarized as follows: 

1. Derive the threshold equation of the CD system. 



2. Approximate the threshold equation by a stepwise curve satisfying the N and M 
constraints. 

3. Write an expression for the cost in terms of the curve parameters. 

4. Minimize the average cost with respect to the curve parameters. 

Let us illustrate how the algorithm w'orks for the case of detection of a knowm 
signal in gaussian noise. 

3. An Example: The Known Signal in Gaussian Noise 
Consider Figure 3.2 when y t and y 2 are given by 



into two decision regions. Since there are as many as 2^ M fusion methods, checking all 
of them will consume a very long time even for small values of N and M. A suboptimal 



A (yj ,y 2 ) = C 



(3.37) 



H n : y. = n. 

0 J i i 

H 1 : y i = a i + n i > i = 1(2 



(3.38) 



67 




Fusion 

Center 



uo 

T 



Figure 3.2 Configuration B. The Team of Two 
Quantizers and a Fusion Center. 
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Figure 3.3 Quantized Threshold Curve. 
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where the a/s are positive constants and N = [n : n 2 ] l is a gaussian random vector of 
zero vector mean with covariance matrix: 



It is required to design the N-level and the M-level quantizers Qj and Q 2 
and the decision rule y where 



to minimize the average decision cost. 

Procedure following the QD algorithm 

The threshold equation of the CD problem has been shown in Chapter II to have the 
form: 



1. The CD curve is a straight line in the y 2 plane. 

2. Possible stepwise approximations for the threshold equation are shown in 
Figure 3.4 . We notice that in Figure 3.4 a and c the two quantizers have the 
same number of quantizer levels. While in Figure 3.4 b and d one quantizer has 
one more level than the other. From Chapter II, we can expect that the 
constant C will decide the superiority of a or c and of b or d. We shall 
consider optimum parameters of Figure 3.4 a and b. Similar treatment can be 
considered for Figure 3.4 c and d. In Figure 3.4 a the point Xj =-co while Tj 
is finite. In Figure 3.4 b Xj = -<x> and Tj = so . 

3. The probability of detection of the decision rule of Figure 3.4 a is given by 




(3.39) 



Y = Y Iq X Y 2q -> (0,1) 



(3.40) 



(aj -p a 2 )Vj +(a 2 +p 3j )y 2 = (aj 2 +a 2 2 -p a : a 2 )/2 + (l-p 2 )log(C). (3.41) 




(3.42) 



and P f is given by 




(3.43) 
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For the detection rule of Figure 3.4 b P d is given by : 



p d = X $ +1 /H i } dy i dy * 

and P f is given by 
Nj X 00 

p f - ^ i+ 1 Jr K y i- y 2 /H ° > dy i ^2 

4. Necessary conditions for optimality of parameters of the curve in 
are: 

f^VVWdy, 

A (T, ) = C — i ,i= 1,2 N 



and 



f^ 1 fTy 2 /X,,H 0 )dy 2 

A (Xj ) = C — » ,i= 2,3 N . 

1/' Hyj.'Xj.H,) dy 2 



For Figure 3.4 b ,the optimality conditions are 



f^yj/Wdyi 

A (Tj ) = C -U ,i-2,3,...,N 

J * +I li:y 1 /T i ,H l )dy 
A i 



and 



(3.44) 

(3.45) 

Figure 3.4 a 

(3.46) 

(3.47) 

(3.48) 
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,i=2,3,...,N . 



( 3 . 49 ) 



jl 1 ' 1 dy 2 

A (X, ) = C — i 

Jx' 1 dy 2 

The last two equations are exactly the same as the necessary conditions for 
optimizing detection using a Primary Decision Maker and its quantized second opinion 
for the same signals in gaussian noise. Recall that the information available at the 
PDM is more complete than that available at the fusion center of two quantized 
observations. Yet the two problems have the same solution. This is a proof of the 
following lemma. 

Lemma 3.2 

Optimum detection of known signal in gaussian noise using two quantized observations 
of N and N + 1 levels is equivalent to optimum detection using the first quantized 
observation and the second continuous observation. 

Lemma 3.2 is applicable to any case with a monotonic likelihood ratio. This can be 
easily proved by writing the necessary conditions of optimality for the two 
configurations. A special case of Lemma 3.2 is that of N = 2. It corresponds to the 
tandem configuration of two detectors in a Distributed Detection Network (DDN) [10]. 
The "downstream" detector (decision maker) makes its decision based on its own 
observation and the "upstream" detector's decision. 

D. NUMERICAL SOLUTION FOR THE SYSTEM PARAMETERS 

It is of interest to compare the four sets of equations {(3.24 ),(3.26)}, 
{(3.32), (3.33)}, {(3.46 ),(3.47)} and {(3.48), (3.49)} with that of Lloyd and Max [18,23] for 
minimum distortion quantizer parameters. 

Max's trial and error algorithm to solve this set of nonlinear equations can be 
used. However Max's algorithm is very time consuming [24]. We have used instead 
the method of successive substitutions with an initial guess satisfying 

X 2 < X 3 < ....^ X N (3.50) 

and put the equations in the form 

Z = G ( Z ) (3.51) 
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a) 




b) 




c) 



d) 



Figure 3.4 Possible Approximations of the Threshold Equation. 
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The kth iteration is then given by 



Zk-GC^i) 



( 3 . 52 ) 



We will devote the next chapter to solving some numerical examples using this method. 
E. SUMMARY 

In this chapter the method of detection using quantized sensor observations has 
been introduced. This method, referred to by QD, can have significant performance 
improvement compared to the distributed detection algorithm (DD) with only 
marginally more demand on the communication channels. The QD algorithm involves 
approximating the CD threshold hyperplane by a stepwise hyperplane that can be 
spanned with the quantized data and that minimizes the detection cost. 

Also the equivalence between two detection configurations, one with tandem 
connection and the other with hierarchical structure, has been shown. 
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IV. NUMERICAL RESULTS 



In this Chapter some examples are solved numerically using the QD algorithm. 
First the detection of known signals in gaussian noise is considered. Next detection of 
signals with exponential distribution is considered. Finally, the algorithm will be 
applied to differentiating between gaussian signals with different variances. 

A. KNOWN SIGNAL IN GAUSSIAN NOISE 

Again consider Figure 3.2 when y t and y 2 are given by 



H 0 



: " n i 

: y. = a. + n.,i- 1,2 



(4.1) 



with aj = 4 and a 2 = 2. The noise vector 

N=[n l n 2 f (4.2) 

is of zero vector mean and with covariance matrix given by: 

- ■ [: 3 

where p is given by 
P = E { n t n 2 } . 

It is required to: 

1. Design the primary decision maker PDM and its N-level Quantizer to minimize 
the average decision cost. We have designated this structure configuration A. 

2. Design the N-level quantizers and Q 2 and the decision rule u Q to minimize 
the average decision cost. We have designated this structure configuration B. 

3. Compare the performance of the two configurations and that of the completely 
centralized system. 

Following the algorithm we have: 

1. The threshold equation for the CD problem given by, 



(4.3) 



(4.4) 
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(a x -p a 2 )y l +(a 2 -p a x )y 2 =(aj 2 + a 2 2 -2p a 2 a 2 )/2 + (l-p 2 )log(C) (4.5) 



a straight line in the yj y 2 plane. 

2. Figures 3.4 a and 3.4 b show the decision regions for configuration A and 
configuration B respectively. 

3. Probability of detection P dl and probability of false alarm P^ of PDM are 
given by: 



dl 



7$ exp( - y > 2/2) - fxr 3l) } dy - 



(4.6) 



and 



P n =. ^ J Xi + 1 — !- exp(-y 2 /2) erfc I dy. 

n i=^l J Xj V2 ti FWl |V(l-p 2 ) / n 



Also P d2 and P^ of configuration B are given by: 



d2 



-M 



2 r 



; +1 v^ exp( - y ‘ 2/2) erfc { Tjc 7u^ :il2 } dy * 



(4.7) 



(4.8) 



and 



P 



f2 




r X i + i_L 

J Xj V 2 it 



exp(-yj 2 12) 



erfc 



T k ' P Zl 

V (i-p 2 ) 




(4.9) 



4. For configuration A equations of the quantizer interval end points and 
corresponding PD M's thresholds for the gaussian case are given by 



A (X„ ) - C 



. \ V (i-P 2 ) / \ V(1-P 2 ) 



afc f T k-i- a 2 -p < x k- a i) crfc r T k -h-t> < x k - a . 

\ VO-P 2 ) / \ V (i-p 2 ) J 



,k= 2,3, N 



and 
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,k= 1,2, N. (4.11) 



A (T t ) - C 



erfc f WP T t -) crf: f VPV ) 

erfc \ V 0-P 2 ) / lV(l-p 2 )J 

erfc 1 . erfc f X k • \ • p (T k . - . a 2 . ) 

\ V 0-p 2 ) / \ VO-P 2 ) J 



For configuration B the quantizer end point intervals X's and T's are given by: 



A(X k ) = C 



, Vd-p 2 )/ 1 V(i-p 2 )/ 






k - a 2 • P ( X k * a i ) 
V (l-P 2 ) 
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,k=2,3, N 



and 



A (T k ) = C 



erfc/i±l±iVe 

X V (1-P 2 ) / \ V (1-P 2 )/ 

cr r c / X k+1 ~ a l ~P ( F k* a 2^ crfc/ Xk ' a l ~ P — 2 } \ 

1 V (i-p 2 ) j" \ V (i-P 2 ) J 



,k= 2,3, N 



We have solved the system of equations of the two configurations using the 
method of successive substitution for N = 2, 3, 4, 5 and 6 . Figure 4.1 shows the 
receiver operating characteristics ROC for the two configurations for p = 0, for different 
values of N. The ROC for the CD system is also shown. The effect of p is illustrated 
in Figure 4.2. The figure shows ROC curves for Configuration A for different values of 
N and for p = 0 and 0.25. Figure 4.3 shows the average cost of Configuration B and 
CD vs. C, for different values of N. The relation between the cost of detection for 
Configuration B vs. the number of quantization levels is shown in Figure 4.4 . The 
figure shows the exponential decay of the detection cost as the amount of information 
available at the fusion center increases. 

The following results are noted from the curves. 

1. Configuration A has better ROC curves than Configuration B. The 
performance difference is large for N = 2 but gets smaller as N increases. 

2. Both performances converge to that of the CD in a uniform manner. 

3. As the correlation coefficient increases the performance difference decreases. 



(4.12) 



. (4.13) 
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Figure 4.1 ROC Curves for Configuration A , 
Configuration B and CD. 
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Figure 4.2 ROC Curves for Configuration A for p =0 and 0.25. 
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Figure 4.3 Average Detection Cost for Configuration B and CD. 
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Figure 4.4 Average detection cost for Configuration B vs. N. 
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4. As N increases the average detection cost gets smaller and tends to that of the 
CD. Since the number N reflects the mutual information between the input and 
the output of the quantizers, the relation between the performance degradation 
and information delivered to the fusion center is strong. 

B. SIGNALS WITH EXPONENTIAL DISTRIBUTIONS 

Consider again Figure 3.2 . Let y t and y 2 have the following distributions: 

. H 0 : fty ) = X Q exp(- X Q y ; ) (4.14) 

and 

H,: fty ) - X, expt-^ y, ) ,i= 1,2 (4.15) 

and assume that X 1 is less than X Q . It is required to design the quantizers and fusion 
rule that minimize the average decision cost. 

Following the QD algorithm we have: 

1. The CD threshold equation is given by 



y, + y 2 = c, 

where Cj is given by 



C l 



Mi1 2 c 

X q - Xj 



(4.16) 



(4.17) 



The CD threshold equation is a straight line in the first quadrant. 

2. Figure 4.5 shows possible approximations of the threshold equation. For N=2 , 
the symmetry suggests equal detector thresholds. For N ^ 3 let us fix Xj and 
T n to zero. 

3. The probability of detection and probability of false alarm P d and P f are given 
by 



■2 



[ exp(-X 1 X; )- exp(-Xj X i+ 1 ) ] exp(-Xj T. ) 



(4.18) 



and 
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(4.19) 



N 

p f = z [ exp(-X 0 Xj )- exp(-X 0 X. + 1 ) ] exp(-X Q T. ) 
i= 1 



4. Writing an expression of the average cost in P d and P f as before and minimizing 
with respect to X k ,k = 2,3,..N and T k , k=l,2,.. N-l one obtains the set of 
equations 



exp{(X Q -Xj )T k } = 




exp(-X n X. )-exp(-X n X , , ) 

C F 0 k; — 0 - + 1 - ,k= 1,2,... N-l 

expf-Xj X k )-exp(-X 1 X k+1 ) 



(4.20) 



and 



exp{(X Q -Xj )X k } = 




exp(-X Q T k )-exp(-X Q T k+1 ) 
exp(-X 1 T k )-exp(- X Q T k+J ) 



,k= 2,3,... N. 



(4.21) 



This set of equations Have been solved by the method of successive 
substitutions for X Q = 2, Xj =1, and for N = 2,3,4,5 and 6. A FORTRAN 
program to calculate the quantizer parameters is given in Appendix D. 

Figure 4.6 shows ROC curves for the quantized as well as the CD systems. The 
average detection cost is shown in Figure 4.7. 

We note the following: 

1. The largest performance improvement occurs when we switch from N = 2 to N 
= 3 ( i.e. only less than one more information bit per detection). 

2. The performance curves ( ROC(N) ) and ( R(N) } converge uniformally to the 
performance of CD 

C. GAUSSIAN SIGNALS WITH DIFFERENT VARIANCE 

Consider again the structure of Figure 3.2 . Let sensor observations yj and y 2 be 
independent, identically distributed gaussian random variables of zero mean. However, 
under H 0 , Varty )=<t q 2 , 
and 

under Hj, Varty ) = <jj 2 ,i= 1,2. For specificity, let 
= 1 and <Tj = V 2 . 

Quantized sensor observations are sent to the fusion center to decide which of the 
hypothesis is true. It is required to design the quantizers Q t and Q 2 as well as the 
fusion rule to minimize the average decision cost. 
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yi 




a) N = 2 





0 N = 4 



d) N = 5 



Figure 4.5 Approximation of the Threshold equation for 
Different Values of N, for Exponential Signals. 
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Figure 4.6 ROC curves for Exponential Signals. 
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Figure 4.7 Average Detection Cost for Different 
Values of N for Exponential Signals. 
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Following the QD algorithm we have: 

1. The CD system decision rule is a likelihood ratio test. The CD detector declares 
Hj is true if 

y , 2 +y 2 2 < (l/2)Iog[<r, 2 /(<T„ 2 C)]/(ff 0 ' 2 -ff,' 2 )•* (4.22) 

otherwise it will declare H Q is true. The threshold equation is the circle 

y , 2 + y 2 2 - V ( 4 - 23 > 

where R 0 2 is the right hand side of inequality (4.22 ). 

2. Possible approximations of the CD threshold equation are shown in Figure 4.8 . 

3. Figure 4.8 a corresponds to 3-level quantizers. The corresponding probability of 
detection and probability of false alarm are given by; 

P d (3)=[erf(-X/ff 1 )] 2 (4.24) 

and 

P f (3)=[erK-X/<r 0 )) 2 (4.25) 



where y t and y 2 are subdivided by the points X and -X. For the 5-level 
quantization approximation of Figure 4.8 b, the probability of detection and 
probability of false alarm are given by 



P d (5)= erfIX 3 /ff 1 ) {2erf(X 2 )- er«X 3 ) ) (4.26) 

and 

P f (5) = erKX 3 /<r 0 ) {2erf(X 2 /c Q ) -erl(X 3 /<r Q ) } (4.27) 

where X 2 , X 3 ,-X 3 and -X 2 defme the the quantization intervals of both Vj 
and y 2 . 

4. Inserting P f (3) and P d (3) into R in (3.3 ) and minimizing R with respect to X 
gives 
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(4.28) 



A (X ) = C-i- — 

(T 0 er flX /<Jj ) 

Also inserting P d (5) and ?^5) into (3.3 ) and minimizing R with respect to X 2 
and X 3 gives; 



<r, erf(X, /<y n ) 

A (X, ) = C — ! 3 - 0 - . 

2 S er^/orj) 



(4.29) 



A (X ) = c A- er « X 2 i°o > • er « X 3 / g n ) 
' 3 <t q erf(X 2 ,'<7 1 ) - erf(X 3 /<T 1 ) 



(4.30) 



Solution of these implicit equations in the quantizer parameters can be carried 
out by the method of successive substitution. The FORTRAN program to calculate 
them for any value of and ffj is given in Appendix F. 

Figure 4.9 shows the average detection cost vs. C for 3-level and 5-level 
quantizer systems. Detection cost of CD is also shown. The figure shows that the 
detection cost decreases dramatically using 5-level quantizers in comparison to 3-level 
quantizers. The cost of the CD system is only slightly lower than that of the 5-level 
quantizers. 

Similar procedures can be carried out for the case of correlated observations. The CD 
curve in this case is an ellipse with principle axes passing through the origin. It can be 
approximated in a similar way as the circle. 

D. CONCLUSION 

The above examples show the uniform convergence of the Quantized Detection 
Algorithm to the Centralized Detection Algorithm. The Distributed Detection 
Algorithm is a special case of QD. It follows that Quantized Detection is an efficient 
utilization of bandlimited communication channels. 
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y2 




b) 



Approximation with 5 Levels 



Figure 4.8 Possible Approximation Of the Threshold Equation. 







Figure 4.9 Average Detection Cost. 



V. THE CASE OF VECTOR OBSERVATIONS 



A. INTRODUCTION 

In the previous two chapters the QD problem for the case of scalar sensor 
observations was solved. It is now time to extend the QD algorithm to the case where 
each local observation is a vector Yj . The QD algorithm can be applied as long as the 
corresponding sufficient statistic for the centralized detection problem can be divided 
into local statistics to be quantized. Let us consider the gaussian case and put it in the 
previous framework. 

B. QUANTIZED DETECTION WITH VECTOR OBSERVATIONS 

For the structure of Figure 5.1 the observations at locations 1 and 2 are given by 



H n : 7. - N. 

o —i — i 



and 




(5.1) 



Let us denote the observation vector bv 7 



1 = 



(5.2) 



The noise vector N , given by 



N = 



(5-3) 



is multivariate gaussian with zero vector mean and covariance 




R = 



(5.4) 
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Figure 5.1 Vector Observations. 



, R 2 and R [2 are the covariance matrices of the noises at locations 1 and 2 and their 
common covariance matrix. The signal vector A is given by 




The CD system decides that Y belongs to Z ^ if [6] 

exp { (-1/2) [ (Y -A)'/?' 1 (Y -A) -Y 'fl' 1 Y ] } £ C . (5.6) 

The CD threshold equation can be written in the form 

A 'R' [ Y = log(C) -(1/2) A'R' 1 A. (5.7) 

Using the block matrix inversion lemma [25], (5.7 ) can be written in the form 

a Yj +p Y 2 = log(C) -(1/2) A'#' 1 A . (5.8) 

In (5.8 ) a and P are given by 

« = A ' 1 ( ff 1 - fl 12 5 2 ’ 1 5 21 )‘ 1 

-A 2 (R 2 -R n R j * 1 R 2l y [ R 21 flj’ 1 (5.9) 

and 

P =-a , 1 (5 1 -5 12 5 2 ‘ 1 5 21 )‘ 1 5 12 5 2 ' 1 

+ A’ 2 (R 2 -R 2 l R l ' 1 R l 2 Y l ■ (5.10) 

Denoting 

h = « Yj (5.11) 

and 
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*2“ P h 



(5.12) 



(5.8 ) becomes 

lj + 1 2 = log(C)-(l/2)A'5‘ 1 A (5.13) 

where ^ and 1 2 are bivariate gaussian with zero vector mean under hypothesis H Q . 
Under hypothesis Hj their vector mean is 




Covariance of lj and 1 2 is given by 



(5.14) 



Cov(lj ,I 2 ) 



a’ R j a 




P 

P *fl 2 p 



In (5.15 ) p is given by 



P 



[• 



a E l2 P V | (« 2, a V (P R, P ) 



-l 



■1 



(5.15) 



(5.16) 



The distributed signal processing is to form local linear combinations l t and 1 2 , then 
quantize them as before. This processing is also shown in Figure 5.1. 



C. SUMMARY 

In this Chapter it is shown that the QD algorithm can be extended to the case of 
sensor vector observations. An application is the case of high quality local area 
communication and lower quality long distance communications. In this case sensor 
observations in local areas are gathered at a local processor to form the local sufficient 
statistics. Quantized local statistics are then sent to the global far away processor for 
fusion. 
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VI. OPTIMUM ESTIMATION USING QUANTIZED SENSOR 

OBSERVATIONS 



A. INTRODUCTION 

In the previous part of this thesis there are situations in which a group of 
observers make local decisions that, taken in combination determine the overall 
performance of a system. The observers may or may not be interconnected. However, 
even when they are, for a variety of considerations such as limitations on 
communications bandwidth, transmitter power, security, or perhaps the very nature of 
the observers themselves, only decisions may be interchanged between them and not all 
the observations upon which their decisions are based [1,5,26-33]. 

Another case of interest concerns the encoding of high resolution measurements 
for transmission between observers using a small number of bits. Here a remote 
observer must decide which of N possible discrete values best represents his 
observation. A second observer is to combine his local observations with the discrete 
data from the first in an optimum manner. In this chapter we consider the problem of 
regeneration of a remote sensor observation using its quantized representation and a 
local observation. The design of the quantizer at the remote sensor location and the 
optimum linear estimator to combine the quantized data with the local observation to 
minimize the expected mean square estimation error will be considered. Generalization 
of the results to the vector case is also shown. 

B. THE LINEAR MINIMUM MEAN-SQUARE ESTIMATE OF Y 1 

Consider the structure of Figure 6.1 in which the observation Vj is quantized 
into yj by a quantization rule y 




( 6 . 1 ) 



The quantized data y lq is sent to sensor S, site. 

The linear minimum mean square estimate of the observation Vj from y lq and y 2 is 
shown in (Appendix F) to be 



A 




( 6 . 2 ) 
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Figure 6.1 Estimation Using Quantized Sensor Observations. 
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where we have assumed that yj and y 2 are random variables with zero mean and 
variances 



E{y 2 } = <r 2 i ,i=l,2 (6.3) 

and correlation 

P =E(y 1 y 2 }/<7 1 <t 2 (6.4) 

The scalar quantities r\ and ji are parameters of the quantizer and are given by 



n = 






Q k 2 )/<V 



and 



P 



= (V N P 

( ^l k 



Qk C k >/<v 



(6.5) 



( 6 . 6 ) 



N is the number of quantization levels and C k and P k are given by 

C k = (Jv k+ S ) dyj )/P k (6.7) 

A k 

and 

P k = X! k + 1 ftyi ) dy x . (6.S) 

A k 

Q k is the k^ quantization value and Q k e[.X k ,X k + 1 ]. In (6.7) and (6.8), X k 
,k= 1,2, ....N, are the quantization interval end points, with 
Xj =-oo and X N + 1 = co . 

It is required to design a quantizer that minimizes the mean square estimation error. 
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The expected mean square error is given by [34] 



E((y, -y lq ) 2 } = V -E{y l r }E(Z I' }•' E{r y, } 
where 7 isthe vector 

Z-tyjq 



(6.9) 



( 6 . 10 ) 



Equation (6.9 ) can be written in the form (Appendix D) 



E(e 2 ) = ^ 2 (l-P 2 )(l-co )/(l-p 2 to ) 



( 6 . 11 ) 



where to is given by: 

to=p 2 /n- (6.12) 

A plot of E(e 2 }/a ] 2 vs. to is shown in Figure 6.2 for p 2 = 0, 0.25, 0.5, 0.75. The figure 
shows that the mean square error is decreasing with to. Recall that the criterion is to 
minimize the mean square error. 

Equivalently the problem now is to maximize to over all quantization rules where 



to 




(J sI P 1 c QtCk,2 



(6.13) 



Appling the Cauchy Inequality [35] to the numerator yields 

< |L , p k c k > 2 5 k £, p « v yl, p k c k 2 > (« i4 > 

with equality if and only if 

Q k = C k . (6.15) 
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Figure 6.2 Relative Mean Square Estimation Error vs. oo. 



99 



0.6 



Therefore 



co < 



l 




r=i 



p k Qk 2 



(6.16) 



gives an upper bound of co. Equation (6.15) says we maximize co , and thus minimize 
E{e 2 } by making the quantization level Q k equal to the conditional mean of yj given 
that y } lies in the kth quantization interval. This is one of the conditions 
characterizing the classical Lloyd-Max quantizer [18,23.] There remains the problem of 
how to pick X k ,k=l,2,...N, so that the upper bound of co in (6.16) is maximum. 
Notice that the upper bound of co is r\. Therefore, the optimum quantizer will be a 
Lloyd-Max quantizer if we prove that maximizing t] over all choices of the set of 
points {X k }, k= 1,2,. ..N, is equivalent to minimizing the distortion E^y^-y^ ) 2 }. Since 

E«y r y lq ) 2 } - >1 (6-17) 

= v(i-n) 

then maximizing r| will minimize the distortion E((yj-y lq ) 2 } and vice versa. Since the 
Lloyd-Max quantizer is the optimum quantizer for minimum distortion it follows that 
it is also optimum for our problem. Accordingly choose X k 's such that [23,18], (see 
also Appendix G) 



V _ + Qk-1 

Xk 2 



,k= l,2,...N. 



(6.18) 



Equations (6.15) and (6.1S) along with (6.7) completely design the quantizer [23,18]. 
Parameters of the Lloyd-Max quantizer can be calculated efficiently using the method 
of successive substitution (Appendix G). Values of E[e 2 } / cr A 2 vs. N are listed in Table 
2 for p =0, 0.25, 0.5, 0.75. The table shows the exponential decay of the MMSE as the 
number of quantization levels increases. 

Table 3 shows a comparison between the average number of bits per sample used in 
this system and another method in which the Maximum Output Entropy (MOE) 
Quantizer [36] is used. Huffman coding [37] is assumed for both quantizers. 
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TABLE 2 



MINIMUM MEAN SQUARE ERROR VS. THE 
NUMBER OF QUANTIZATION LEVELS 



N p 


0 


0.25 


0.5 


0.75 


2 


0.3634 


0.3548 


0.3241 


0.2477 


4 


0.1175 


0.1166 


0.1131 


0.1021 


8 


0.0345 


0.0345 


0.0342 


0.0330 


16 


0.0095 


0.0095 


0.0095 


0.0094 


32 


0.0025 


0.0025 


0.0025 


0.0025 


64 


0.0006 


0.0006 


0.0006 


0.0006 


128 


0.00016 


0.00016 


0.00016 


0.00016 



TABLE 3 

COMPARISON OF THE AVERAGE NUMBER OF BITS IN 
THE MMSE AND THE MOE SYSTEMS 



N 


2 


4 


6 


8 


Optimum 

System 


1 


1.989 


2.4768 


2.8842 


MOE 


1 


2 


2.667 


3 
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C. CONCLUSION 

The trade off between performance and communication is clear from Table 2. 
For p =0.5 the relative MMSE is 0.75 without communication. This corresponds to 
substituting CO = 0 in (6.11 ). The relative MMSE decreases to 0.32 using one 
information bit per sample. The relative MMSE is 0.11 using two bits/sample. It is 
only 0.03 using 3 information bits/sample (N=8) and is 0.00016 using 7 bits/sample. 
We also notice that for high number of quantization levels the estimation error is 
approximately equal to the the quantization error. This means that the estimator 
depends mainly on y lq for fine quantization. For coarser quantization the estimator 
depends heavily on y 2 to reduce the MMSE. Table 3 shows that the designed system 
has considerable reduction in the number of bits per sample compared to the MOE 
quantizer system. 

D. GENERALIZATION TO THE VECTOR CASE 

In this section we will consider regeneration of a random vector Y l from its 
quantized version Yj and a correlated continuous scalar y 2 . As an application 
consider a sensor S-, monitoring the activities of N stations. Due to some 
considerations, perhaps of safety nature, only simple sensors can be placed near the 
stations. Because of other considerations, such as limited bandwidth communication 
channels, only quantized sensor measurements can be sent to the monitor. Specific 
examples can be the case of monitoring the states of a target in a far field or the 
positions of N targets in a multitracking problem [38,39]. Another example is to 
monitor the radiation levels outside of N nuclear reactors. A third example is 
monitoring the activities of N enemy transmitters. 

Let us design the quantizers at the N sensor sites and the estimation rule at the 
monitor site so as to minimize the mean square error of each component of Y . Let Yj 
,the sensor observation vector be given by; 



where Vj. is the sensor observation ,j= 1,2,...,N. We will assume that components of 
Y are independent, i.e. 



Under the above conditions, also y^ and y t . are conditionally independent for i * j, so 



= ty 11 y 12 -y 1N ] t . 



(6.19) 




( 6 . 20 ) 
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ffrii /Z lq ,y 2 )=fly u /y 1Iq ,y 2 ) - i= 1 - 2 N * 

The MMS estimate of Yj given Yj is given by 



( 6 . 21 ) 



Zl = E { /Z lq ,y 2 } . 



Or 




LE{y 1N /y 1Nq ,y 2 L 



Let us denote the error vector by E , so 



( 6 . 22 ) 



(6.23) 



E = 



t e l e 2 




(6.24) 



where e ; is the error in estimating y. ,i = 1,2 N. The MMS error covariance matrix is 



E {EE 1 } =E((Y 1 -y i )(I 1 -Y 1 ) t }. (6-25) 

The trace of the error covariance matrix is given by 

trace(E [EE 1 }) =V N E(e.2 } (6.26) 

i= 1 

where 

e i - E (yn /^liq >■ ( 6 - 27 > 

Minirhizing the trace of the covariance matrix in (6.26 ) is accomplished by minimizing 
each summand alone since every summand is nonnegative. Now assuming Linear 
Minimum Mean Square Estimation, the problem of minimizing E (e^ } implies using 
the Lloyd-Max Quantizer to quantize y n as was shown previously. 
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In conclusion the Linear Minimum Mean Square Estimate of the observation 
vector Y implies using the Lloyd-Max quantizers at the sensor sites and the same linear 
combining considered in the scaler case at the central processor. 
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VII. SUMMARY, RESULTS, CONCLUSIONS AND SUGGESTIONS FOR 

FUTURE RESEARCHS 



A. SUMMARY 

This Thesis begins by listing some reasons why Distributed Signal Processing is 
more practical than Centralized Signal Processing. The status of Distributed Detection, 
an important case of Distributed Signal Processing, and its complexity are reviewed in 
Chapter I. . 

Chapter II deals with the problem of optimum fusion of local decisions into a 
global decision. The relationship between the optimum fusion rule and the ratio of 
error cost is shown. The dependance of the performance of the optimum fusion rule on 
the correlation coefficients between sensor observations is throughly analyzed. For 
higher values of the correlation coefficients the Distributed Detection system is shown 
to reduce to the detector of the highest signal-to-noise ratio. 

A compromise technique between Centralized Detection and Distributed 
Detection, Quantized Detection, is suggested in Chapter III. The main issue of that 
chapter is to control the degree of centralization according to the communications 
channel constraints. The Quantized Detection technique replaces local detectors by 
quantizers and sets a global fusion rule that approximates the centralized decision rule. 
The algorithm matches the other techniques at extreme limits. 

Chapter IV contains some specific applications of the Quantized Detection 
Algorithm for detection problems. A significant performance improvement is attained 
by replacing Distributed Detection with Quantized Detection with three quantization 
levels (one and half information bits per sample vice one information bit per sample). 

Chapter V considers applicability of the Quantized Detection Algorithm to the 
case of vector observations. In this case local sufficient statistics are quantized in the 
same way as before. 

Chapter VI deals with the regeneration of sensor observations from their 
quantized versions and another correlated observation. The local quantizers and the 
optimum linear data fusion are designed. We arrive at the following results and 
conclusions. 
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B. RESULTS 



1. Detection with Distributed Sensors 

a. Optimum Fusion Rules in Distributed Detection 

The optimum fusion rule depends on the ratio of costs of different types of 
detection errors. For high cost of false alarm relative to the cost of missing the target 
the AND fusion rule is better than the OR fusion rule, and vice versa. 

The performance of the optimum fusion rule depends on the degree of 
correlation between sensors. The performance degrades as the correlation coefficient 
increases. The worst performance of the optimum fusion rule is at and above a critical 
value of the correlation coefficient p cr . In that region of correlation the best system 
employs only the detector of higher signal-to-noise ratio, ignoring the lower signal-to- 
noise ratio sensor entirely. The performance of the Distributed Detection system 
improves as the signal-to-noise ratio imbalance between sensors increases. However 
there is a large performance difference between the Centralized Detection and the 
Distributed Detection for values of the correlation coefficient above p cf . 

Below p cr the performance of the Distributed Detection system improves as 
the correlation coefficient gets smaller. The best performance (lowest detection cost) of 
the Distributed Detection system is achieved at p = -l. Recall that the Centralized 
Detection system has perfect detection at p =-l. This is due to the efficient use of the 
information contained in two observations of positive signals and anticorrelated noise 
samples. 

The large performance difference between Centralized Detection and 
Distributed Detection systems is due to the loss of information in the local detection 
processes. As a remedy to the performance degradation in Distributed Detection we 
have introduced the Quantized Detection Algorithm. 

b. Quantized Detection 

There is a great improvement in the system performance using Quantized 
Detection with three quantization levels in comparison to Distributed Detection. This 
performance difference between Quantized Detection and Distributed Detection 
decreases as the correlation between sensors increases. 

The Quantized Detection algorithm is applicable to the case of vector 
observations and waveform observations. In those cases, the local sufficient statistics 
are to be quantized at the local processor and transmitted to the central site for fusion. 
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The Quantized Detection algorithm is implemented by the quantizers as 
local processors and a fusion rule, suggested by the Centralized Detection decision rule, 
at the central site. The quantizers used in the Quantized Detection algorithm are 
designed to minimize the detection cost. 

2. Minimum Mean Square Estimation in Distributed Sensor Systems 

Minimum mean square estimation in Distributed Sensor Systems involves the 
classical Lloyd-Max minimum distortion quantizers at the local levels and linear 
processing at the global central level. A faster iterative algorithm to calculate the 
Lloyd-Max quantizer parameters is the method of successive substitution. It also has 
more accurate results than previously reported techniques. 

C. CONCLUSIONS 

We conclude the following: 

1. Global optimization of the Distributed Detection implies picking the fusion rule 
and corresponding local decisions that minimizes the detection cost. 

2. The optimum fusion rule in Decentralized Detection depends on the correlation 
coefficient , the a priori probabilities and the ratio of costs. 

3. For optimum fusion of two local unbalanced decisions there is a particular 
value of p that decides the optimum fusion rule. 

4. For p ^ p cr OR fusion is better for higher cost of missing target while AND 
fusion is better for higher costs of false alarm. 

5. For p > p cr the optimum fusion rule is to ignore the sensor of lower signal-to- 
noise ratio and optimize the decision of the higher signal-to-noise ratio sensor. 

6. The poor performance of Distributed Detection compared to Centralized 
Detection is due to the loss of information at the local levels. 

7. The Quantized Detection system matches the Distributed Detection system and 

the Centralized Detection system for the two extreme conditions of 

quantization. As the number of quantization levels increases the Quantized 
Detection converges to Centralized Detection. 

8. The Quantized Detection algorithm has a tremendous improvement in 

performance over Distributed Detection even with only 3 quantization levels. 

9. The performance difference between Quantized Detection and Distributed 

Detection increases as the correlation of the observations gets smaller. 

10. In case of linear Centralized Detection threshold equations in the observation 
space, Distributed Detection and Centralized Detection are special cases of 
Quantized Detection. 

11. The Quantized Detection algorithm can get the maximum allowable 

performance in the presence of communication constraints. 



107 



12. The Quantized Detection algorithm can be applied to arbitrary distributions for 
the observations. 

13. The method of successive substitution is applicable to the design of many types 
of quantizers. It has a simple programming procedure and very accurate results. 

D. SUGGESTIONS FOR FUTURE RESEARCH 

The following are some areas the Quantized Detection algorithm can extend to: 

1. Optimum detection using quantized sensor observations for the case of 
unknown signal in noise. 

2. Detection of M-ary phenomena using quantized sensor data. 

3. Utilizing the Quantized Detection algorithm over noisy channels. 

4. Illustration of the relation between the complexity in some suitable units and 
the amount of information delivered to the fusion center. 

5. Utilizing the Quantized Detection algorithm to meet the Neyman-Pearson 
criterion. 

6. Extension of Distributed Detection and Quantized Detection to more than two 
sensors with correlated observations and unequal SNR's. 

7. Development of general principles for parsing fusion rules given a Centralized 
Detection surface in N-dimensional space. 

8. Application of the Quantized Detection method to target detection, classification 
and tracking using distributed sensors. 
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APPENDIX A 

PROBABILITY OF DETECTION AND PROBABILITY OF FALSE 
ALARM OF THE PRIMARY DECISION MAKER 

Given that the Primary Derision Maker (PDM) receives Qj (the quantization 
level of the Consultant observation yj), and that its own observation is y 2 , its 
observation space is divided into two decision subspaces Z^ and Z Q y Let us denote the 
conditional probability of detection and probability of false alarm given Qj by P d j and 
P^ respectively. P dj . and P fj are given by: 

P dj = Pr( Declare Hj /y lq =Q ] ,Hj is true) (A.l) 



and 



P f] = Pr(Declare Hj /y lq =Q ] ,H Q is true). 



(A. 2) 



These can be expressed as: 




(A. 3) 



and 




(A. 4) 



or equivalently as , 




(A. 5) 



and 




(A. 6) 
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But ,y 2 /Q- ,H ; ) is given by [40] 



fty .y 2 /Qj.Hj)= 



{ 



fty ,y 2 / H i )/Pr(Qj /Hj ) 

0 otherwise 



(A.7) 



where Pr(Qj /K ) is the probability of the j A quantization level of yj under hypothesis 
R . It is given by: 

Pr(Q:/Hj) = J ^ + ^ yi /Hj) dy 2 ,j = 1,2, ...N ,i-0,l (A.S) 

X i 

The probability of detection and probability of false alarm are now 



P 



d 




Pr (Qj /Hi )P dj 



(A. 9) 



and 



Pf “S', PrtQj .H„ )P n 



(A. 10) 



Inserting (A.5 and (A.6 ) into (A. 9 ) and (A. 10 ) yields 




flj r y 2 /H i ) dyj dv 2 

ij 



(3.4) 



and 



P 



f 




r X i + 1 

J X. 

1 



GO 

L . Jtyi.y 2 /H 0 ) dyi dy 2 

■2 6 ^lj 



(3.5) 



where Z l - is the decision region Z 1 given that y A e [ Xj ,Xj + 1 ] . 
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APPENDIX B 

PROGRAM LISTING TO CALCULATE PARAMETERS OF THE N AND 

THE (N + 1)-LEVEL QUANTIZERS 



In this appendix we give a program listing to calculate the parameters of the N 
and the (N+ l)-level quantizers in configuration A 



C******************************************************************* 

C THIS PROGRAM CALCULATES THE OPTIMUM N-LEVEL AND (N+l) -level 
C OUANTIZER PARAMETERS 

C FOR A SYSTEM OF TWO QUANTIZERS AND THEIR FUSION CENTER. 

C THE RATIO OF COSTS C RANGES FROM 0.1 AND 10 . 

C THE VALUE OF N =2, 3 ,4, 5 AND 6. 

C THE PROGRAM USES THE MODIFIED METHOD OF SUCCESSIVE SUBSTITUTIONS . 

Q**Jt*jt*K*jtj'***K****j'jt***Jt**7t*j'KJt********K****jt**'kJ''kjt***’kjt*’k7*-k'k**'k'k*'k 

REAL*8 X(9) ,T(9) , XX(9),TT(9),C,S1,S2,A.B,A12,A21,R,C1(20),R1(2) 
1 , AERR,RERR , ERROR , PD ,PF,PD1(20,9,6) ,PF1(20,9,6) ,PD2 

I, PF2,PDC,PFC,X2,R3(20,2,6) , XI 1 , DCADER , FI , F2 , APD , APF , A1 , A2 , AP 
INTEGER K,N, I ,P ,N1 ,N2 ,M, J,MX 

EXTERNAL F1,F2 
COMMON Xll , R 

DATA Cl/10 . 0D0 , 9 . 0D0 , 8 . 0D0 , 7 . 0D0 , 6 . 0D0 , 5 . 0D0 ,4 . 0D0 , 3 .0D0 , 2 . 0D0 , 

II. 50D0, 

11.0D0, . 90D0 , . 80D0 , .70D0, .60D0, .50D0, .40D0, .30D0, . 20D0 , 0 . 10D0/ 
Al=l . 0D0 
A2=2 . 0D0 
Sl=l . 0D0 
S2=l .0D0 
R1 ( 1 )=0 . 600D0 
R1 ( 2 )=0 . 8000D0 
AERR=0 . 0D0 
RERR=0 . 00010D0 

********************************************* ******** 



INITIAL GUESS 
XX(1 )=-9 . 50000D0 
XX(2)=-1.9000D0 
XX 3)=-1.50000D0 
XX(4)=-0 .89000D0 
XX( 5 )=00 . 50000D0 
XX(6 )=0 . 09000D0 
XX(7 )=00 . 99000 DO 
XX( 8 )=01 . 50000D0 
XX(9)=2.09000D0 
TT( 1 )=4 . 67000D0 
TT(2)=3.89000D0 
TT 3)=2.67000D0 
TT(4)=2.29000D0 
TT(5)=1.90000D0 
TT(6)=1.49000D0 
TT( 7 )=1 . 09000D0 
TT(8)=00 .50000D0 
TT{9 }=0 . 19000D0 

C*******X;fcw ********************************************** 



C THE FOLLOWING INITIAL VALUES OF T'S CORRESPOND TO THE CASE 
C OF CORRELATION COEFFICIENT GREATER THAN A1/A2 
TT(9 )=4 . 67000D0 
TT(8)=3.89000D0 
TT(7)=2.67000D0 
TT( 6 )=2 . 29000D0 
TT( 5 )=1 . 90000D0 
TT(4)=1 .49000D0 
TT(3)=1 . 09000D0 
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TT(2)=00 . 50000D0 
TT (l )=0 . 19000D0 
DO 500 N=2 ,2 

WRITE(9 ( 60)N 
DO 600 1=6,9 

C INPUT VALUE (S) OF THR CORRELATION COEFFICIENTS 

R=0.10D0*DFLOAT(I) 

WRITE (9 , 61 )R 
C DO 200 M=1 , 20 

M=1 

C=C1(M) 

WRITE (9 , 61)C 



c INPUT MAXIMUM NUMBER OF ITERATIONS HERE 

MX=250 
K=0 

5 K=K+1 

IF (K .GT. MX) GO TO 10 
IF (K . GT . 1 ) X(N)=XX(N) 

IF (K . GT . 1 ) T(N)=TT(N) 

IF (K .GT. 1)X(N)=XX(N) 

IF (K .GT. 1)T(N)=TT(N) 

A=(S1*XX(2)-R*S2^TT(1) )/ (S1*S2*DSQRT ( ( 1 . 0D0-R**2 ) *2 . 0D0) ) 
A12=(S2*A1-R*S1*A2)/(S1*S2*DSQRT((1.0D0-R**2)*2.0D0)) 
A21=(S1*A2-R*S2*A1 ) / { S1*S2*DSQRT ( (1 . 0D0-R**2) *2 . 0D0 ) ) 

TT(l)=(A2*A2/2.0D0+DL0G(C)+DL0G((DERFC(A)- 

1 2 . 0D0 ) / (DERFC ( A-A12 ) -2 . 0D0 ) ) )/A2 

T ( 1 ) =TT ( 1 ) 

A=(S1*TT(2)-R*S2*XX(2) )/ (S1*S2*DSQRT ( (l . 0D0-R**2)*2 . 0D0) ) 

B= ( S2*T ( 1 ) -R*S1*XX( 2 ) ) / ( S2*S 1*DSQRT ( ( 1 . 0D0-R**2 ) *2 . 0D0 ) ) 
XX(2)=(A1 X A1/2.0DO+DLOG(C)+DLOG((DERFC(A)- 
1 DERFC (B) )/ (DERFC ( A-A21 ) -DERFC(B-A21 ) )) )/Al 

X(2)=XX(2) 

IF (N .EQ. 2) GO TO 17 
DO 15 P=2 ,N-1 

X(P)=XX(P) 

A=(S1*X(P ) -R*S2*T(P ) ) / (S1*S2*DSQRT ( ( 1 . 0D0-R**2)*2 . 0D0) ) 
B=(S2*X(P+1)-R*S1*T(P) )/(S2*Sl*DSQRT((1.0D0-R**2)*2.0D0)) 
TT ( P ) = ( A2*A2/ 2 . ODO+DLOG ( C ) +DLOG ( ( DERFC ( A) - 
1 DERFC (B) )/(DERFC(A-A12)-DERFC(B-A12))))/A2 

T(P)=TT(P) 

A=(S1*T(P+1)-R*S2*X(P+1))/(S1*S2*DSQRT((1.0D0-R**2)*2.0D0)) 
B=(S2*T(P)-R*S1*X(P+1) )/(S2*Sl*DSQRT((1.0D0-R**2)*2.0D0)) 
XX(?+1 j=(Al*Al/2 . ODO+DLOG(C)+DLOG( (DERFC(A) - 
1 DERFC(B) )/ (DERFC (A-A21 ) -DERFC (B-A21) ) ) ) /A1 

15 CONTINUE 
17 CONTINUE 

TT(N)=(A2*A2/2.0DO+DLOG(C)+DLOG(DERFC((XX(N)-R*TT(N))/ 

1DSQRT ( ( 1 . 0D0-R**2 ) *2 . 0D0 ) ) 

1 /DERFC ( (XX(N)-A1-R*(TT(N)-A2) )/ 

1DSQRT((1.0D0-R**2)*2.0D0)) 

1 ) )/ A2 



C CHECKING THE ACCURACY 
C INPUT REQUIRED PRECISION HERE 
AP=Q . 10d-07 

IF((DABS(T(N)-TT(N) ) .GT. AP ) .OR. (DABS (X(N) -XX(N) ) .GT. 
1 AP) )GO TO 5 



10 CONTINUE 

X(1)=-10.0D0 
X(N+1)=10.0D0 
APD=0.0D0 
APF=0 . 0D0 
DO 81 Q=1 ,N 
A=X(Q) 

B=X(Q+1) 

X11=T(Q) 

APD=APD+0 . 50D0*DCADRE ( FI , A , B , AERR , RERR , ERROR , IRE1 ) 
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IF (IRE1 .NE. 0) WRITE(6,60)IRE1 

APF=APF+0 . 50D0*DCADRE ( F2 , A , B , AERR , RERR , ERROR , IRE 1 ) 
IF (IRE1 .NE. 0) WRITE (6 , 60) IRE 1 
81 CONTINUE 

PD1 (M, I ,N)=APD 
PF1 (M, I ,N)=APF 

R3(M.I.N)=1 .0D0+C1 (M)*APF-APD 



c QUANTIZER PARAMETERS 
DO 120 J=1 ,N 

WRITE (9 , 62) K(J) ,T(J) 

120 CONTINUE 
C WRITE (9, 90) 

C200 CONTINUE 
C WRITE (9, 90) 

C WRITE(9,90) 

600 CONTINUE 
C WRITE(9 , 90) 

C WRITE(9,90) 

C WRITE (9 ,90) 

500 CONTINUE 

DO 501 J=1 , 2 
DO 502 1=1.20 

£ * 3*c & -k & -k 7*c 'k k k k k k k k k k k k k k k k k k k k k k k k k k k k k * k k k k k k k k k k k k 5^ ^ k k k k 



C OUTPUT DETECTION COST 

WRITE ( 10. 61) Cl (I), ( R3(I . J.N) .N=2,6) 



C * OUTPUT PROBABILITY OF DETECTION AND PROBABILITY OF 

C FALSE ALARM FOR DIFFERENT VALUES OF N 

WRITE (8, 61) ( PD1(I,J,N) ,PF1(I,J,N) ,N=2,6) 

502 CONTINUE 

501 CONTINUE 

60 FORMATS IX, I4,4(1X.F15.8)) 

C90 FORMAT (2X ‘CON A x***'k*£********kkkkkkkkkkkkkkkkkkkkkk 1 ) 

61 FORMAT (IX*, 10(1X,F6 .4) ) 

62 FORMAT (1X,2(1X,F15.8)) 

STOP 

END 

FUNCTION FI (X) 

REAL*8 X,F1,A1,A2,R,X11,F11,F12 
COMMON Xll ,R 
Al=l . 0D0 
A2=2 . 0D0 

F11=DEXP(-(X-A1)**2/2.0D0)/ 

1DSQRT ( 8 . 0D0*DATAN ( 1 . 0D0 ) ) 

F12=DERFC( (X11-A2-(X-A1)*R)/ 

1 (DSQRT ( 2 . ODO* ( 1 . 0D0-R**2 ) ) ) ) 

F1=F11*F12 

RETURN 

END 

FUNCTION F2 (X) 

REAL* 8 X,F2,R,X11,F11,F12 

COMMON Xll ,R 

F11=DEXP ( -X**2/2 . 0D0 ) / 

1DSQRT ( 8 . 0D0*DATAN ( 1 . ODO ) ) 

F12=DERFC((X11-X*R)/ 

1DSQRT ( 2 . ODO* ( 1 . 0D0-R**2 ) ) ) 

F2=F11*F12 

RETURN 

END 
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APPENDIX C 

PROGRAM LISTING TO CALCULATE PARAMETERS OF THE TWO 

QUANTIZERS 



In this appendix we give a program listing to calculate the parameters of the two 
N-level quantizers in Configuration B. 

Qkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk kkkkkkkkkkkkkkkkkkkkJ&tkkkkkkkkk 

C THIS PROGRAM CALCULATES THE OPTIMUM N-LEVEL QUANTIZER PARAMETERS 
C FOR A SYSTEM OF TWO QUANTIZERS AND THEIR FUSION CENTER . 

C THE CORRELATION COEFFICIENT IS ASSUMED TO BE LESS THAN A1/A2. 

C FOR THE CORRELATION COEFFICIENT IS GREATER THAN A1/A2 THE 
C THE PROGRAM NEEDS SLITE MODIFICATIONS ACCORDING TO THE QD ALGORITHM. 

C THE RATIO OF COSTS C RANGES FROM 0.1 AND 10. 

C THE VALUE OF N =2, 3, 4, 5 AND 6. 

C THE PROGRAM USES THE MODIFIED METHOD OF SUCCESSIVE SUBSTITUTIONS . 

REAL*8 X(8) ,T(8) ,XX(8),TT(8) , A1 , A2 , SI , S2 , R,T12 ,T21 , AA2 , A,B 
1 , AERR , RERR , ERROR , PD , PF , PD1 ( 20 , 9 , 6 ) , PF1 (20 , 9 , 6 ) , PD2 
1 ,PF2 , PDC , PFC ,C1 (20) , A21 , A12 , C,X2 

I , X11 , DCADER, F1,F2,APD,APF,R3(20,2,6) 

INTEGER K,N,I,IER1,IER2,M,P,Q,L 
EXTERNAL F1,F2 

DATA Cl / 10 . 0D0 , 9 . 0D0 , 8 . 0D0 , 7 . 0D0 , 6 . 0D0 , 5 . 0D0 , 4 . 0D0 , 3 . 0D0 , 2 . 0D0 , 

II. 50D0, 

11.0D0, . 90D0 , . 80D0 , .70D0 , . 60D0 , . 50D0 , . 40D0 , .30D0 , . 20D0 , 0 . 10D0/ 
COMMON Xll.R 

C INPIT SIGNALS HERE 
Al=4 . 0D0 
A2=2.0D0 

C INPIT VARIANCE HERE 
Sl=l . 0D0 
S2=l . 0D0 

Q**j'****7s*x*ysx***7s*jr**********&********jr****-k-k'k’k-k-k-k'k-k-k-kj:-k'k'k'k-k'k'k 

C INITIAL GUESS OF THE PARAMETERS 
XX(1)=-0.50000D0 
XX(2)=0 .89000DO 
XX( 3 )=01 . 50000D0 
XX(4)=2 . 89000D0 
XX(5)=03.50000D0 
XX(6)=4.09000D0 

C INITIAL VALUES OF T'S FOR CORRELATION COEFFICIENT GREATER THAN A1/A2 
TT(1)=-0.67000D0 
TT(2)=0 .89000D0 
TT(3)=01.67000D0 
TT(4)=2 . 89000D0 
TT( 5 )=03 . 50000D0 
TT(6)=4.89000D0 

Qkkkkkkkkk k-xkkkkkkkkkkkk *********** ****** ********** A*** A* A***** 

C INITIAL VALUES OF T'S FOR CORRELATION COEFFICIENT LESS THAN A1/A2 
C PUT TT( 1 ) > TT(2)> >TT(N) 

Qkkkkkk kkkk A kkkkkkkxkkkkkkkkkkk7rk ***************************** kkk 

AERR=0 . 0D0 

RERR=0 . 00010D0 

DO 500 N=1 , 5 

DO 11 1=1,2 

R=DFLOAT(I-1)*0 .250D0 

DO 20 M=1 , 20 

C=C1 (M) 

K=0 

5 K=K+1 
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non 



25 



15 

16 



10 



WRITE(6 ,60)K,T1 ,T11 , T2 ,T22 
IF (K .GT.100) GO TO 10 
DO 25 L=1,N 
T(L)=TT(L) 

X(L)=XX(L) 

CONTINUE 

XX ( 1 ) = ( A1*A1 / 2 . ODO+DLOG ( C) +DLOG ( DERFC ( (T ( 1 ) -R*X ( 1 ) ) / 
1DSQRT(<1.0D0-R**2)*2.0D0) J 
1/DERFC((T(1)-A2-R*(X(1)-A1))/ 
1DSQRT((1.0D0-R**2)*2.0D0)) 

D)/A1 

IF (N .EQ. 1) GO TO 16 

A21=(S1*A2-R*S2*A1 )/ (S1*S2*DSQRT( (1 . 0D0-R**2)*2 . 0D0) ) 
A12=(S2*A1-R*S1*A2)/(S1*S2*DSQRT((1.0D0-R**2)*2.0D0)) 
' P=1 ,N-1 



A=(S1*X(P)-R*S2*T(P) 

B=(S2*X(P+1)-R*S1*T(P 




1 . 0D0-R**2)*2 . 0D0 ) ) 

(_S_2*S1*DS_QRT ; ((1 . 0D0-R**2) * 2 . 0D0 ) ) 

A2 

r**2) *2 . 0D0 ) ) 
r**2)*2 „0D0) ) 



CONTINUE 



CONTINUE 



TT(N)=(A2*A2/2 . ODO+DLOG (C)+DLOG( DERFC ( (X(N) -R*T(N) )/ 
DSQRT ( ( 1 . 0D0-R**2 ) *2 . 0D0 ) ) 



IDS 

1/DERFC{(X(N)-A1-R*(T(N)-A2))/ 



1 . 0D0-R**2 ) *2 . ODO ) ) 



.GT. 



1DSQRT( 

1) )/A2 

IF( (DABS (T(N)-TT(N)) . 

1 0 . 10D-05) )GO TO 

CONTINUE 
X(N+1 )=10 . ODO 
APD=0 . ODO 
APF=0 . ODO 
DO 81 Q=1 ,N 
A=X(Q) 

B=X(Q+ 

Xll 



0.10D-05) .OR. (DABS (X(N) -XX(N) ) .GT. 
5 



4 ( 5 + 1 ) 

l=f(0) 



APD=APD+0 . 50D0*DCADRE ( FI , A , B , AERR , RERR , ERROR , IRE1 ) 
IF (IRE1 .NE. 0) WRITE (6 , 60 ) IRE1 
APF=APF+0 . 50D0*DCADRE (F2,A,B, AERR , RERR , ERROR , IRE1 ) 
IF (IRE1 .NE. 0) WRITE (6,60) IRE1 
81 CONTINUE 

PD1 (M, I ,N)=APD 
PF1 (M, I ,N)=APF 

R3 (M, I ,N)=1 . 0D0+C1 (M)*PF1 (M, I ,N) -PD1 (M , I ,N) 

C OUTPUT SYSTEM PARAMETERS 
DO 120 J=1,N 

WRITE (9 , 62 ) X( J) ,T( J) 

120 CONTINUE 

WRITE (9 ,61)C,T(1) ,T(2) ,APD,APF 
WRITE (8 , 61 ) APD , APF 

WRITE (6 ,61)C,X(1) ,X(2) ,APD,APF,PDC,PFC 
20 CONTINUE 
11 CONTINUE 

C WRITE (9, 90) 

WRITE(8,90) 

500 CONTINUE 

Q^-k'k'k'k-k'k-k-k-k-k-k-k-k-k-k'k-k-k-k-k-k-k-k-k-k'k-k-k'k-k-k-k-k-k-k-k-k-k-k-k-k-k'k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k-k 

C OUTPUT AVERAGE COST 

DO 501 J=1 , 2 
DO 502 1=1,20 

WRITE (10, 61) Cl (I) , ( R3(I , J,N) ,N=1,5) 

Qk'k'k'k'k-k-k'k'k-k-k'k'k'k'kTi-k'k'k'k'k-k'k'k'K-k'k-kTs-k-k-k-k-k-k'k'kii-k-k-kii-k-k-k-k'k-k-k'k'k-k-k-k-k'k-k-k-k 

C OUTPUT PROB. OF DETECTION AND PROB. OF FALSE ALARM 
WRITE (8, 61) ( PD1 ( I , J ,N) , PF1 (I , J ,N) ,N=1,5) 

502 CONTINUE 
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WRITE (8, 90) 

501 CONTINUE 

60 FORMAT (1X,I2,9(1X,F6.4)} 

90 FORMAT ( 2X 'CON B ************************************') 

61 FORMAT ( IX ^ 10 (1X,F6 .4) ) 

62 FORMAT (IX, 2 (IX, FI 5. 8) ) 

STOP 

END 

FUNCTION F1(X) 

REAL*8 X,F1,A1,A2,R,X11,F11,F12 
COMMON Xll ,R 
Al=4 . 0D0 
A2=2 . 0D0 

F11=DEXP (- (X-Al )**2/ 2 . 0D0 )/ 

1DSORT ( 3 . ODO*DATAN ( 1 . ODO ) ) 

Fl2=DERFC( (X11-A2-(X-A1)*R)/ 

1 (DSQRT(2 . 0D0*(1 . 0D0-R**2) ) ) ) 

F1=F11*F12 

RETURN 

END 

FUNCTION F2(X) 

REAL*8 X, F2 ,R, Xll , Fll , F12 

COMMON Xll ,R 

F11=DEXF ( -X**2/ 2 . ODO) / 

1DSQRT ( 8 . ODO*DATAM ( 1 . ODO ) ) 

F12=DERFC( (X11-X*R)/ 

1 DSQRT ( 2 . ODO* ( 1 . 0D0-R**2 ) ) ) 

F2=F11*F12 

RETURN 

END 
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APPENDIX D 

PROGRAM LISTING TO CALCULATE PARAMETERS OF THE TWO 
QUANTIZERS FOR THE CASE OF EXPONENTIAL DISTRIBUTIONS 

Q *•&*********************** 

C THIS PROGRAM CALCULATES THE OPTIMUM N-LEVEL QUANTIZERS OF TWO SEN- 
C SOR OBSERVATIONS OF EXPONENTIAL DISTRIBUTIONS 
C TO MINIMIZE A GLOBAL SYSTEM RISK FOR FUSION 
C SEE CHAPTER IV 

(2 ’k'k'k'k'kjcjc-kjc’kjtjcjc-kJcjtjc'k'kjc&jc’k'kjc 

REAL*8 X(8) ,T(8) ,XX(8) ,TT(8) ,PD ,PF ,PD1 (20 , 9 , 6) ,PF1(20,9,6) ,PD2 
1 , PF2 , PDC , PFC , Cl (20 ) , C ,X2 ,X11 . APD , APF ,R3 (20 ,2,6) ,AL,BL,Y 
1 ,APC(20 ,2) ,AFC(20,2) , R4(20 , 2) , AD2 , AF2 ,RR2 , AP 
INTEGER K,N, I , IER1 , IER2 ,M,P ,Q,L ,MX 

DATA Cl/50 . 0D0 ,40 . 0D0 , 30 . 0D0 , 20 . 0D0 , 15 . 0D0 , 10 . 0D0 , 9 . ODO , 

18 . 0D0 , 7 . 0D0 , 6 . 0D0 , 5 . 0D0 , 4 . 0D0 , 3 , 0D0 , 2 . 0D0 , 1 . 50D0 , 

11 • ODO , . 90D0 . . 80D0 . .70D0, .60D0/ 

C ************************* 

C INITIAL VALUES OF QUANTIZER PARAMETERS 
C FIRST QUANTIZER 

XX(1)=00.00000D0 
XX(2)=0 .89000D0 
XX(3 )=01 . 50000D0 
XX(4)=1 . 89000D0 
XX( 5 }=02 . 50000D0 
XX(6 )=3 . 09000D0 
C SECOND QUANTIZER 

TT(1 )=3 . 090 00 ODO 
TT(2 )=2 . 50000D0 
TT(3 )=01 .89000D0 
TT(4)=1 .50000D0 
TT(5)=00 .89000D0 
TT(6 )=0 . 00000D0 
DO 500 N=3 , 6 
DO 11 1=1,2 

C PARAMETERS OF THE EXP. DISTRIBUTIONS 
AL=DFLOAT(I)*0.50D0 
DO 20 M=1 , 20 
C=C1 (M) 

DO 55 P=1,N 

TT ( P ) =DLOG ( 2 . 0D0*C ) ^DFLOAT (N-P+l ) /DFLOAT (N) / AL 
XX(N-P+1 )=DLOG(2 . 0D0*C)*DFL0AT (N-P+l )/DFLOAT(N)/AL 
55 CONTINUE 
K=0 

5 K=K+1 

DO 89 PP=1,N 

T(PP)=TT(PP) 

X(PP)=XX(PP) 

89 CONTINUE 

QA****A**7C*7t ********** 

C INPUT MAXIMUM NUMBER OF ITERATIONS 
MX=200 

IF (K . GT . MX ) GO TO 10 
C XX(l)=DLOG(2.0D0*C)/AL-TT(l) 

XX( 1)=0 . ODO 

TT ( 1 ) = ( DLOG (DEXP ( - AL*XX ( 1 ) ) +DEXP ( - AL*XX ( 2 ) ) ) +DLOG ( 2 . ODO *C ) ) / AL 
DO 15 P=2,N-1 

XX ( P ) = ( DLOG ( DEXP ( -AL*TT ( P ) ) +DEXP ( - AL*TT ( P- 1 ) ) ) +DLOG ( 2 . ODO *C ) ) / AL 
TT ( P ) = ( DLOG ( DEXP ( - AL*XX ( P ) ) +DEXP ( - AL*XX ( P+1 ) ) ) +DLOG (2 . 0D0*C) )/AL 
15 CONTINUE 

XX (N ) = ( DLOG ( 2 . 0D0*C) +DLOG ( DEXP (TT (N ) ) +DEXP ( - AL*TT (N- 1 ) ) ) ) / AL 
C TT (N)=DLOG(2 . 0D0*C)/AL-XX(N) 

TT (N)=0 . ODO 
C ACCURACY CHECKING 
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C INPUT PRECESSION HERE 
AP=0 . 10d-05 

IF( (DABS (XX (N) -X(N) ) .GT. AP) .OR. (DABS(TT(l)-T(l)) .GT. 

1 AP } ) GO TO 5 

10 CONTINUE 
WRITE (8 .60) K 

BL=2.0D0*AL 
APD=0.0D0 
APF=0 . 0D0 
DO 81 Q=1,N-1 

APD=APD+DEXP ( - AL*TT (Q))*( DEXP ( -AL*XX(Q) ) -DEXP ( - AL*XX <0+1 ) ) ) 
APF=APF+DEXP ( -BL*TT(Q) )* (DEXP ( -BL*XX (Q) ) -DEXP (-BL*XX(Q+1 ) ) ) 
81 CONTINUE 

APD=APD+DEXP ( -AL*X(N) ) 

APF=APF+DEXP ( -BL*X(N) ) 
APC(M,I)=(1.0D0+DLOG(4.0D0*C))/(4.0D0*C) 

AFC (M , I ) = ( 1 . ODO+DLOG ( (4 . ODO*C) **2 ) ) / (4 . ODO*C) **2 
PD1 (M, I ,N)=APD 
PF1 (M, I ,N)=APF 

R3 (M, I ,N)=1 .0D0+C1 (M)*PF1 (M . I ,N)-PD1 (M, I ,N) 
R4(M,I)=1.0D0+C1(M)*AFC(M,I)-APC(M,I) 

DO 120 J=1 ,N 

Q A jk -k -k -k "k -k A -k A ■k x -k * -k •k * Jr ^ x * A ■k & A A * A A & 

C OUTPUT QUANTIZER PARAMETERS 
WRITE (9,62) XX ( J ) , TT ( J ) 

120 CONTINUE 

WRITE(9 , 90) 

20 CONTINUE 

11 CONTINUE 
WRITE (9 ,90) 

C WRITE (9, 90) 

WRITE (8, 90) 

500 CONTINUE 

DO 501 J=1 , 2 

DO 502 1=1,20 

AP2=1.0D0/(2.0D0*C1(I)) 

AF2=AP2**2 

RR2=1 . 0D0+C1 ( I ) *AF2-AP2 



C OUTPUT DETECTION COST 

WRITE(10,61)C1(I) ,RR2, ( R3(I,J,N) ,N=3 ,6) ,R4(I , J) 

Q & ^ j* * x "k h x ^ x x tc x -k & x x x rr is x rr & it * x 



C OUTPUT PROB. OF DETECTION AND PROB. OF FALSE ALARM 

WRITE ( 8 , 61 )AP2 , AF2 , ( PD1 (I , J ,N) ,PF1 (I , J ,N) ,N=3 , 5) , APC( I , J) 
1 , AFC ( I , J) 

502 CONTINUE 

WRITE (8 ,90) 

501 CONTINUE 

60 FORMAT (1X,I6,9(1X,F6.4)) 

90 FORMAT (2X, 'CON B i ^ 

61 FORMAT 1X',10(1X,F6.4)) 

62 FORMAT (IX, 3 (1X,F15.8)) 

STOP 

END 
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APPENDIX E 

PROGRAM LISTING TO CALCULATE PARAMETERS OF THE TWO 
QUANTIZERS FOR EXAMPLE 3, CHAPTER IV 



C THIS PROGRAM CALCULATES THE OPTIMUM QUANTIZER PARAMETERS OF TWO 
C N-LEVEL QUANTIZERS IN ORDER TO MINIMIZE A GLOBAL SYSTEM RISK FOR 
C DETECTION OF SIGNALS WITH DIFFERENT VARIANCE. N=3 , N=5. 

C SI = SIGNAL VARIANCE UNDER HI 
C SO = SIGNAL VARIANCE UNDER HO 
C T1 = QUANTIZATION POINT FOR N=3 ( T1,-T1 ) 

C XI ,X2 QUANTIZATION POINTS FOR N=5 (XI ,X2 , -XI , -X2 ) 

C C | <20 ) ARRAY OF RATIO OF COSTS 
c K = Number of iterations. 

REAL*8 T1,T11,S1,S2,R,C,X2,PD,PF,C1(15) ,Z31 ,Z21 ,X3 ,X33 ,X22 
1 , TS , SSS , TTT ( 10 ) , XI , PD3 , PF3 , PDC , PFC , R2 , R3 , RC 
INTEGER K,N,K1 

DATA Cl/01 . 0D0 , . 900D0 , . 800D0 , . 700D0 , 0 . 60D0 , 0 . 50D0 , 0 . 40D0 . 

10 .30D0 , 0 . 20D0 , 0 . 10D0 , .090D0, .080D0, . 070D0 , 0 . 060D0 , 0 . 050D0/ 
Sl=l . 0D0 
S0=DSQRT ( 2 . 0D0 ) 

c INITIAL VALUES OF THE QUANTIZER PARAMETERS 
Tl=01 . 15800D0 
Xl=01 . 15800D0 
X2=00 . 15800D0 
WRITE (6 ,60)K,S1,S0,C 
WRITE (9 ,60)K,S1,S0,C 
WRITE ( 8, 60 )K, SI ,S0,C 
SSS=(1.0D0/S0**2-1.0D0/Sl**2)/2.0O0 
DO 101 N=2 , 2 
DO 100 1=1,15 
C=C1(I) 

K=0 

5 K=K+1 

IF (K .GT. 100) GO TO 10 
T11=T1 

TS=(DL0G(S1/S0)+ 

1DL0G(C)+DL0G( (DERF (T1/DS0RT(2 . ODO) 
1/S0))/(DERF(T1/DSQRT(2.0D0)/S1)))*DFL0AT(N-1) )/SSS 
C IF (TS .GT. O.ODO) T1=DSQRT(TS) 

T1=DSQRT(DSQRT(TS) ) 

IF( (DABS (Tl-Tll ) .GT. 0.10D-05)) GO TO 5 
CONTINUE 

10 TTT (N)=T11 
K1=0 

55 K1=K1+1 

IF (K1 .GT.100) GO TO 15 
X33=X3 



C 



15 



X22=X2 

X2=(DL0G(S1/S0)+ 

1DL0G ( C ) +DLOG ( ( DERF ( X3/DSQRT ( 2 . ODO ) 

1/SO) ) / (DERF(X3/DSQRT(2 . ODO) /SI ) ) ) )/SSS 
IF (TS .GT. O.ODO) T1=DSQRT(TS) 

X2=DS0RT(X2) 

Z30=DERF ( X3/DSQRT ( 2 . ODO ) /SO ) 

Z20=DERF(X2/DSQRT(2.0DO)/S0) 

Z3 1=DERF ( X3/DSQRT ( 2 . ODO ) /SI ) 

Z21=DERF (X2/DSQRT(2 . ODO)/ SI) 

X3=(DLOG(S1/SO )+ 

1DL0G ( C ) +DLOG ( (Z30-Z20)/(Z31-Z21) ) )/SSS 
X3=DSQRT ( DABS ( X3 ) ) 

IF(((DABS(X2-X22 .GT. 0.10D-05)) .0R.(( DABS(X3-X33) .GT. 
10 . 10D-05 ) ) ) GO TO 55 
CONTINUE 
TTT (N)=T11 
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TS=-DL0G(S0**2/S1**2/C)/SSS 
PDC=1 . ODO-DEXP ( -TS/ ( 2 . ODO*S 1*S1 ) ) 

PFC=1 . ODO-DEXP ( -TS/ (2 . ODO*SO*SO ) ) 

RC=1 . ODO+C*PFC-PDC 
WRITE (9 , 60)K,T1 ,T11 ,X2 ,X22 ,X3 ,X33 
WRITE (6 ,60)K,T1 ,T11 
PD=DERF(T11/DSQRT(2 . ODO)/Sl )**N 
PF=DERF (T11/DSQRT(2 . ODO)/SO)**N 
R2=l . ODO+C*PF-PD 

PD3=(DERF (X22/DSQRT(2.0D0)/S1)*2 .ODO-DERF (X33/DSQRT(2 .ODO)/S1) ) 
1 *DERF(X33/DSORT(2.DDO)/S1) 

PF3=(DERF (X'22/DSQRT (2 . 0D0)/S0)*2 . ODO-DERF (X33/DSQRT(2 . ODO )/SO) ) 
1 *DERF (X33/DSQRT(2 .OD0)/S0) 

R3=1.0DO+C*PF3-PD3 

WRITE (8 , 60)N,PDC,PFC,PD,PF,PD3 ,PF3 

WRITE ( 10 ,60)N,C,R2 ,R3,RC 

C IF ((I .EQ. 1) .OR. (I ,EQ. 10 )) WRITE ( 10 , 60)N, C,R2 ,R3 ,RC 

WRITE(6,60)N,C,TTT(N) ,PD,PF,PD3 ,PF3 

100 CONTINUE 

101 CONTINUE 

60 FORMAT (1X,I3,6(1X,F10.7)) 

STOP 

END 
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APPENDIX F 

LINEAR MINIMUM MEAN SQUARE ESTIMATE OF Yj 

Having Y =[ y 2 ] l the LMMS estimate of and the corresponding mean 
square error are given by [34]: 



y x = Ety £ }E{Y £ K 1 Y (F.l) 

and 

E{e 2 }= Hfy, 2 )-E{y, Y‘ } E{Y Y‘ J' 1 E{Y y, } (F.2) 

where 



E{y 1 X t } = [E{y 1 y 1 } E{ yi y 2 } ] 



and 



E{Y £ ] = 



E{y 2 lq } E{ 
E{y 2 y lq ) £{ 



yiq ?2 H 

^ 2 ) J 



The entries of these matrices are: 



E (yi ^lq ) “jS'j’i Q i Ef y l /y lq “Qj ) 

N 

= y p. q. c. 

\=\ j « 

= fl (7. 2 



Ni 



E{y 2 yi q }^Ij p j Qj E(y 2 /Yiq = Qj) 



(F.3) 



(F.4) 



(F.5) 



(F.6) 



but 
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E{y 2 /y lq = Qj } = E{{y 2 / yj }/y lq = Qj } 



(F.7) 



For the case where and y 2 are jointly gaussian, we can write 

E(y 2 y lq > = P «i « 2 ^ Pj Q,. Cj/ff, 2 (F.8) 

E (y 2 y iq } = p ^ ( F - 9 ) 



E <yn 2 >iL p i% 



(F.10) 



<v n • 



Inserting these in (F.l ) and (F.2 ) and performing matrix multiplications yields 



v i = |o-P 2 



)p y la + p — L (n -p 2 )y 
q *2 



y 2 J /(n -p 2 p 2 ) 



( 6 . 2 ) 



and 



E{e 2 ) = <r 1 2 (l-p 2 )(l-(0 )/(l-p 2 to ) 



(611) 



where (0 is given by: 



co = p 2 ,% 



( 6 . 12 ) 



APPENDIX G 

SOLUTION OF THE LLOYD-MAX QUANTIZER PARAMETERS BY 
THE METHOD OF SUCCESSIVE SUBSTITUTION 

1. INTRODUCTION 

The minimum distortion quantizer parameters [18,23], as well as parameters 
based on other criterion such as quantizers for signal detection [41], minimum risk 
quantizers and quantizers for LMMS estimation error dealt with in this thesis, can be 
solved by Max's trial and error technique [18]. There are also many other 
approximation methods to calculate the quantizer parameters [42], [43] and [44]. 

In this Appendix we apply the method of successive substitution and its 
modifications [19] to solve for the Llovd-Max quantizer parameters. It is more accurate 
and computationally more efficient than the previously reported methods. It is shown 
to easily generate 7 bit (128 level) optimum quantization. 

2. STATEMENT OF THE PROBLEM 

The Lloyd-Max minimum mean square distortion quantizer problem deals with 
transforming a random variable X of differentiable probability density function f(x) 
into the N-level discrete random variable Y. 



Differentiating D with respect to Xj and y. { yields the following necessary conditions of 
optimality : 




(G.l) 



The optimum parameters minimize the distortion D 




( 0 . 2 ) 



with 





(G.3) 
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yj -(J Xi+1 xf(x)dx)/(J Xi+1 f(x)dx ) ,i- 1,2, -N (G.4) 

X. Xj 

a set of simultaneous equations of propagating character. That is, if is chosen 
correctly then ^ can be calculated from (G.4 ), y 2 from (G.3 ), x 3 from (G.4 ) and so 
forth [18]. In this case the value ofy N calculated from (G.3 ) must agree with its value 
calculated from (G.4 ) with x N + 1 — <x>. This was the core of Max's trial and error 
algorithm; to pick a value for and calculate the parameters up to and including y N , 
which must agree with the value of y N calculated from (G.4 ), otherwise, to pick 
another value of Vj. Let us put the system of equations in the form 

Z=G(Z) (G.3) 

where Z is a 2N-1 vector given by: 



2 =[>’1 x 2 y 2 y N ] £ 



(G.6) 



and apply the iterative substitution 



-new = - (-old > < G - 7 > 

with a suitable initial guess. The convergence is guaranteed if d G k Id Zj is sufficiently 
small for every k,j = 1,2,...,2N-1 [19]. From (G.4 ) 



a a !d y. =[(x j + 1 -y. )f(x. + 1 ) + (y. - Xj )f(x. )]/(2P j ) 



(G.8) 



where is the probability the input of the quantizer is in the interval. 



P. =f X i + 1 f(x)dx. (G.9) 

J x. 

j 

The numerator in (G.8 ) is an approximation of the integral in (G.9 ) by the 
trapezoidal rule with the subdivision [x^ .y^ , Xj + 1 ], so the value of the derivative is 
very likely less than one. Also, substituting for yj and y^ + 1 in (G.3 ) from (G.4 ) and 
differentiating with respect to Xj it is easily to show that 
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(G.10) 



d G ] Id x. «(y. -x. )flx. )/(2P.) + (x. -y H )flx. )/2P H 

which is less than ( d Gj jd y.). The method can be more efficient if we use the updated 
values in the same iteration. In this modification of the method the best current 
values of the parameters are used. This choice may also enhance convergence. The 
method also avoids the tedious calculation of the upper limit of the integral to solve for 
the next x. in (G.4 ). 

3. NUMERICAL RESULTS 

We have solved for the quantizer parameters for a gaussian random variable of 
zero mean and unit variance for several values of N up to 128. Also the mean square 
error D and the output entropy (-£ k P k log 2 (P k )) have been calculated. The results 
presented in Table 4 show that in several cases Max's results, which were only 
available up to N= 36, are not accurate in the last digit. 

Key to Table 4 

The numbering in the table is as follows. 

1. For N even, each table begins with the (N/2+ 1)^ parameters. In this case the 
(N/2+ 1)^ value of x is zero. 

2. For N odd, Each table begins with the (N‘/2 + 2) lh parameters. In this case the 
(N/2 + 2)^ value of v is zero. 

Negative parameters can be calculated from the symmetry relation 



x r 



X 



n-j + 2 



and 



(G.l 1) 



y-i = y n -i+i • < G - 12 ) 

A FORTRAN program to calculate the parameters distortion and entropy 
follows Table 4. The only input to the program is N, the number of quantization levels. 
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TABLE 4 

MAX'S QUANTIZER PARAMETERS FOR THE NORMAL DISTRIBUTION 





N = 2 




N 


= 7 




J 


X 


Y 


J 


X 


Y 


2 


0. 000000 

ERROR 
ENTROPY = 

- __ 


0. 797885 

0. 363380 
1. 000000 


4 -0.280289 

■5 0.280288 

6 0.874362 

7 1. 610758 

"iRROR =’ 


0. 000000 
0. 560577 
1. 188147 
2. 033369 

"o7o44000 


J 


X 


Y 




ENTROPY = 


2. 646931 


2 


-0. 612003 


0. 000000 


N 


= 8 




3 


0. 612003 


1. 224006 














J 


X 


Y 




ERROR 


0. 190174 










ENTROPY = 


1. 535789 


5 


0. 000000 


0. 245094 


: = 






6 


0. 500550 


0. 756005 


N 


= 4 




7 


1. 049957 


1. 343909 








8 


1. 747927 


2. 151946 


J 


X 


Y 




ERROR = 










0. 034548 


3 


0. 000000 


0. 452780 




ENTROPY = 


2. 824865 


4 


0. 981599 


1. 510418 


— ===: 


=========== 


============ 








N 


= 9 






ERROR = 


0. 117482 


j 


X 






ENTROPY = 


1. 911099 


Y 


====z 


2========== 


===========~ 






— — — — — 


N 


= 5 




5 


-0. 221819 


0. 000000 






6 


0. 221819 


0. 443639 


J 


X 


Y 


7 


0. 681217 


0. 918796 






8 


1. 197594 


1. 476392 


3 


-0. 382284 


0. 000000 


9 


1. 865528 


2. 254664 


4 


0. 382284 


0. 764567 




ERROR = 




5 


1. 244357 


1. 724147 




0. 027853 








ENTROPY = 


2. 982695 




ERROR 


0. 079941 


==== = 


===========: 


============ 




ENTROPY = 


2. 202916 


N 


= 10 




~N 


= 6~ 




J~ 


X 


Y 


J 


X 


Y 


6 


0. 000000 


0. 199623 






7 


0. 404740 


0. 609857 


4 


0. 000000 


0. 317716 


8 


0. 833841 


1. 057825 


5 


0. 658911 


1. 000106 


9 


1. 324583 


1. 591340 


6 


1. 446850 


1. 893595 


10 


1. 968218 


2. 345096 




ERROR 
ENTROPY = 


0. 057978 
2. 442789 




ERROR = 

ENTROPY = 


0. 022937 
3. 124584 
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TABLE 4 

MAX'S QUANTIZER PARAMETERS FOR THE NORMAL DISTRIBUTION (CONT'D.) 



N 


= 11 




N 


= 14 




J 


X 


Y 


J 


X 


Y 


6 


-0. 183729 


0. 000000 


8 


0. 000000 


0. 145706 


7 


0. 183729 


' 0. 367458 


9 


0. 293513 


0. 441321 


8 


0. 559913 


0. 752367 


10 


0. 595882 


0. 750443 


9 


0. 965597 


1. 178826 


11 


0. 918039 


1. 085635 


10 


1. 435733 


1. 692639 


12 


1. 276582 


1. 467528 


11 


2. 059193 


2. 425746 


13 


1. 703070 


1. 938612 








14 


2. 281837 


2. 625062 




ERROR 


0. 019220 










ENTROPY = 


3. 253506 




ERROR 


0. 012232 










ENTROPY = 


3. 582050 


N 


= 12 
















N 


= 15 




J 


X 


Y 














J 


X 


Y 


7 


0. 000000 


0. 168438 








8 


0. 340142 


0. 511846 


8 


-0. 136929 


0. 000000 


9 


0. 694313 


0. 876779 


9 


0. 136928 


0. 273857 


10 


1. 081245 


1. 285711 


10 


0. 414310 


0. 554764 


11 


1. 534371 


1. 783030 


11 


0. 702949 


0. 851134 


12 


2. 140733 


2. 498435 


12 


1. 013007 


1. 174879 








13 


1. 360468 


1. 546057 




ERROR = 


0. 016340 


14 


1. 776266 


2. 006474 




ENTROPY = 


3. 371666 


15 


2. 343670 


2. 680866 


N 


= 13 






ERROR 


0. 010737 










ENTROPY = 


3. 676630 


J 


X 


Y 














N 


= 16 




7 


-0. 156887 


0. 000000 








8 


0. 156887 


0. 313773 


J 


X 


Y 


9 


0. 476012 


0. 638251 








10 


0. 812600 


0. 986949 


9 


0. 000000 


0. 128395 


11 


1. 184106 


1. 381263 


10 


0. 258222 


0. 388048 


12 


1. 622890 


1. 864518 


11 


0. 522404 


0. 656759 


13 


2. 214522 


2. 564525 


12 


0. 799550 


0. 942340 








13 


1. 099286 


1. 256231 




ERROR 


0. 014063 


14 


1. 437139 


1. 618046 




ENTROPY = 


3. 480744 


15 


1. 843532 


2. 069017 








16 


2. 400803 


2. 732590 










ERROR = 


0. 009501 










ENTROPY = 


3. 765328 
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TABLE 4 

MAX'S QUANTIZER PARAMETERS FOR THE NORMAL DISTRIBUTION (CONT'D ) 



O 



II 

2J 



I 

I 

i oomr^Of-Hf-itncovo 
i oomovooo^r-i-i 
i o^cor^vocor^corOfH 
I OOOOcnrHCNrorOfHCO 
I OfH^VOfHr^LDOCOLD 
I OCN^vOO»H^COCNOO 



OOOOOf— If— l»— ICNCN 



X I 



tn^cNOf-i^ocooLn 

COco^cocooOLn^f 1 

CNCNCOOr-CNCNLACOr- 

cnooLnocNooco^cr* 

OOCNLAO^rHcOiH^ 
f— I f— I CO m O CO O O ID 



I II 

I II 

I 0000 II 
I II 

I COLD II 
I OCN II 
I OO II 
l OO II 

I C .11 

I O^ II 

I 
I 
I 
I 
I 
I 



II I] 



II 
II 

II o 

II CN 
>< II 
CX II 



i cxo ii 

I OX II 

I I CdH II 

i ooooohhhcncn i cx: z; n 



X 





f 1 


1 MH II II 


1 1 




1 


1 


II 


1 1 




i 


1 


n s i i 




O i OrHCNco^Lnor^ooo i 


ii 


1 1 




1 fHf— If— IfHi— IfHt— If— if— 1 


rH 1 


n 

n 


1 1 




1 


1 


i ii 






1 


1 


i n 






1 O^CNr-r^vDCNfHCN 


1 


i ii 






i OococoocTi^r-o 


1 h- 


1 O 11 






1 OOCOCNLDCO^Or- 


1 O 


i ^ ii 






i ocNOomo^oo 


1 ^ 


1 CO II 






>H 1 O^O^CNCOOOCNOO 


1 00 


1 00 II 


1 >H 




1 OCNH^h-OcoOfHr- 


1 o 


1 ^ II 






1 


I o 


1 00 II 






1 OOOO. — If — Ii — iCN CN 


1 . 


1 0 II 






1 

1 


I o 
1 


1 CO II 

1 M 






1 


1 

1 


1 II 

1 11 






i r^r^C0LncNOOh-o 


1 II 


1 II II 00 






1 OOCOCO^f^fOOO 


1 


1 II 1— 1 






1 H^CT>O^CNOh-00 


1 


1 >< II 




i X * fHfHvoor-oor^Lnco 


1 


1 CX II 


1 X 




i cNCNOcNCOr-oom 


1 CX 


i O II 




1 HHCOcOCOHlCKJi^ 


i o 


1 CX II 






1 


1 CX 


i EH II 






0 

0 

0 

0 

0 

l 

l 

1 

2 


I CX 


f S II 






1 I 
1 


1 IX 


< U II II 
1 II 




| c 


1 

i cnOrHCNco^Lnor- 


1 

1 


1 II £ 

1 11 


1 ^ 




1 « — i r~i i— 1 1 — 1 1 — ; i— 1 1 — 1 1 — | 


1 


1 II 





CNrHOOCOCNr^^h'-^fH 

ooooco^fOf— ir^fHvo 
^r-^Lno^^crir-cri 
COCNOCOcOCOcOVOOOr^ 
O fH CN ^ CO CO CN in r- o 
tHcoinr-cpicNinoocNO 

OOOOOHHHCNCN 



or^o»Hooin»HOor^ 
or-^M00Ln^f o^fco 
OCNOlAOOOiHCOcO 
ooocnr-LDiHOOr^co 
OOHCQLOiHCOOOO 
OCN^OOOfHCOVDOm 

OOOOOHH rH cncn 



rH CN CO LO VO r- 00 O O 
CN 



I 

I 

I 00 CO 

I ooo 

i CNin 

I vOoo 

I Oh 
I OO 

I . . 

j o<tf 

! ii ii 

I 

I >H 

I 0-i 

i CXO 
I OX 

I 

I XX 
l CdW 

I 

I 

I 

I 



o LDCNCNOt^cor^r- 
*0 ^OcOOCNOCNH 
COCOOOfHCOOOOO 
^ o^cocNcnoom 
1-1 ^COcoOO^OOCN 
•-H CO LO 00 • — I CO — ICO 

o OOOtHf— if— ICNCN 



O ^^cNiHcommcN 

o LncNCOooOfHor- 

O LA CO O OCT> CT) N^CO 
o OLAOr-OCNCOcO 
O COCOOcOLDh-cOO 
O CN^E^OCNlAOlA 

O OOOOrHi— If— ICN 



O fHCNCO^LOOr^CO 

•H f — If — ii — IfHi— if— If — If — | 



I 

I 

I 

I CO»H 
I <7^ 
i inh 
i r^r^* 

I OCN 
I OO 
I . . 

I Oco 

I 

I 

I II II 

I 

I (X 

I xo 
i ocx ■ 
i c£Eh 
i cx’z, 

I ww 

I 

I 

I 
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TABLE 4 

MAX'S QUANTIZER PARAMETERS FOR THE NORMAL DISTRIBUTION (CONT'D.) 



I 



i oooomcN<tfr-ir^cor^cNO 
I OOOOCNrHLmOOOCNmm 
I OiQ^coOmmOOmcNr-l 
I OT-HLOCOOOCNrHtHT— ICTiLOLO 
i ooomm^mr-*iHcooOiH 
i OtHcomr^OrH^ma^o 

i 

I OOOOOOrHiHiHiHCNCO 
I 



CO 

CM 



II 

2 
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TABLE 4 

MAX'S QUANTIZER PARAMETERS FOR THE NORMAL DISTRIBUTION (CONT'D.) 



N 


II 

II II 
II 
II 
II 
II 
II 
II 

K) II 

Ui || 




J 


X 


Y 


13 


-0. 083805 


0. 000000 


14 


0. 083805 


0. 167610 


15 


0. 252208 


0. 336806 


16 


0. 423045 


0. 509283 


17 


0. 598128 


0. 686972 


18 


0. 779592 


0. 872212 


19 


0. 970115 


1. 068019 


20 


1. 173279 


1. 278540 


21 


1. 394213 


1. 509886 


22 


1. 640881 


1. 771876 


23 


1. 927050 


2. 082224 


24 


2. 280667 


2. 479110 


25 


2. 778634 


3. 078159 




ERROR = 


0. 004041 





ENTROPY = 


_4. 384064 


N 


= ~ 26 ~ 




J 


X 


Y 



14 

15 

16 

17 

18 

19 

20 
21 
22 

23 

24 

25 

26 



0. 000000 

0. 161536 
0. 324498 
0. 490402 
0. 660961 

0. 838229 

1. 024813 

1. 224230 
1. 441544 
1. 684648 
1. 967207 
2. 316997 
2. 810502 



0. 080593 
0. 242480 
0. 406516 
0. 574288 
0. 747635 

0. 928823 
1. 120803 
1. 327657 

1. 555432 

1. 813865 

2. 120549 

2. 513445 
3. 107559 



ERROR = 
ENTROPY = 



0. 003746 
4. 438843 



N 


= 27 




J 


X 


Y 


14 


-0. 077781 


0. 000000 


15 


0. 077780 


0. 155561 


16 


0. 233975 


0. 312389 


17 


0. 392106 


0. 471823 


18 


0. 553594 


0. 635364 


19 


0. 720073 


0. 804782 


20 


0. 893532 


0. 982281 


21 


1. 076518 


1. 170756 


22 


1. 272495 


1. 374235 


23 


1. 486469 


1. 598704 


24 


1. 726267 


1. 853829 


25 


2. 005461 


2. 157093 


26 


2. 351670 


2. 546247 


27 


2. 840977 


3. 135707 




ERROR 


0. 003483 




ENTROPY = 


4. 491610 


N 


= 28 




J 


X 


Y 


15 


0. 000000 


0. 075012 


16 


0. 150307 


0. 225602 


17 


0. 301760 


0. 377919 


18 


0. 455569 


0. 533219 


19 


0. 613076 


0. 692934 


20 


0. 775854 


0. 858775 


21 


0. 945836 


1. 032897 


22 


1. 125522 


1. 218147 


23 


1. 318326 


1. 418505 


24 


1. 529205 


1. 639905 


25 


1. 765925 


1. 891945 


26 


2. 041975 


2. 192005 


27 


2. 384821 


2. 577637 


28 


2. 870169 


3. 162701 




ERROR = 


0. 003246 




ENTROPY = 


4. 542507 
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TABLE 4 



MAX'S QUANTIZER PARAMETERS FOR THE NORMAL DISTRIBUTION (CONT'D.) 



N 


= 29 




J 


X 


Y 


15 


-0. 072566 


0. 000000 


16 


0. 072566 


0. 145132 


17 


0. 218211 


0 . 291291 


18 


0. 365424 


0. 439557 


19 


0. 515338 


0. 591119 


20 


0. 669236 


0. 747352 


21 


0. 828638 


0. 909923 


22 


0. 995431 


1. 080939 


23 


1. 172074 


1. 263209 


24 


1. 361940 


1. 460671 


25 


1. 569941 


1. 679211 


26 


1. 803788 


1. 928364 


27 


2. 076890 


2. 225415 


28 


2. 416571 


2. 607727 


29 


2. 898177 


3. 188627 




ERROR = 


0. 003032 




_ENTROPY ~ 


_4 L 59 1663 


N 


= 30 _ 




J 


X 


Y 



16 

17 

18 

19 

20 
21 
22 

23 

24 

25 

26 

27 

28 

29 

30 



0. 000000 
0. 140542 
0. 282019 
0. 425412 
0. 571795 
0. 722402 

0. 878709 

1. 042565 
1. 216393 
1. 403530 
1. 608846 
1. 840001 
2. 110332 
2. 447027 
2. 925088 



0. 070155 
0. 210928 
0. 353110 
0. 497714 
0. 645876 
0. 798927 

0. 958490 

1. 126640 
1. 306147 
1. 500912 
1. 716779 
1. 963224 
2. 257440 
2. 636614 
3. 213562 



ERROR 
ENTROPY = 



0. 002839 
4. 639193 



N = 31 



J 


X 


Y 


16 


-0. 068008 


0. 000000 


17 


0. 068008 


0. 136016 


18 


0. 204446 


0. 272876 


19 


0. 342170 


0. 411464 


20 


0. 482101 


0. 552739 


21 


0. 625267 


0, 697794 


22 


0. 772858 


0. 847921 


23 


0. 926315 


1. 004710 


24 


1. 087456 


1. 170202 


25 


1. 258670 


1. 347138 


26 


1. 443261 


1. 539385 


27 


1. 646065 


1. 752745 


28 


1. 874694 


1. 996643 


29 


2. 142413 


2. 288184 


30 


2. 476285 


2. 664385 


31 


2. 950981 


3. 237577 




ERROR = 


0. 002664 




ENTROPY = 


4. 685201 


N 


= 32 




J 


X 


Y 


17 


0. 000000 


0. 065890 


18 


0. 131971 


0. 198052 


19 


0. 264715 


0. 331378 


20 


0. 399039 


0. 466699 


21 


0. 535816 


0. 604934 


22 


0. 676035 


0. 747136 


23 


0. 820850 


0. 894565 


24 


0. 971674 


1. 048783 


25 


1. 130294 


1. 211804 


26 


1. 299072 


1. 386340 


27 


1. 481284 


1. 576228 


28 


1. 681731 


1. 787233 


29 


1. 907981 


2. 028728 


30 


2. 173234 


2. 317739 


31 


2. 504429 


2. 691120 


32 


2. 975926 


3. 260732 




ERROR 


0. 002505 




ENTROPY = 


4. 729784 
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TABLE 

MAX'S QUANTIZER PARAMETERS FOR 



N 


= 64 


J 


X 


33 


0. 000000 


34 


0 066844 


35 


0. 133787 


36 


0. 200932 


37 


0. 268380 


38 


0. 336238 


39 


0. 404616 


40 


0. 473632 


41 


0. 543408 


42 


0. 614079 


43 


0. 685789 


44 


0. 758695 


45 


0. 832972 


46 


0. 908816 


47 


0. 986446 


48 


1. 066112 


49 


1. 148104 


50 


1. 232757 


51 


1. 320468 


52 


1. 411709 


53 


1. 507054 


54 


1. 607210 


55 


1. 713065 


56 


1. 825759 


57 


1. 946794 


58 


2. 078211 


59 


2. 222896 


60 


2. 385143 


61 


2. 571789 


62 


2. 794840 


63 


3. 078922 


64 


3. 492269 



ERROR 
ENTROPY = 



NORMAL DISTRIBUTION (CONT'D.) 



Y 



. 033409 
. 100278 
. 167297 
. 234567 
. 302193 
. 370283 
. 438950 
. 508314 
. 578503 
. 649655 
. 721922 
. 795468 
. 870476 
. 947155 
. 025736 
. 106488 
. 189720 
. 275794 
. 365141 
. 458276 
. 555831 
. 658589 
. 767542 
. 883977 
. 009611 
. 146810 
. 298981 
. 471305 
. 672274 
. 917407 
. 240437 
. 744101 



000644 

710078 



; 4 

THE 

~0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

2 

2 

2 

2 

2 

2 

3 

3 

~ 0 ~ 

5. 
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TABLE 4 

MAX'S QUANTIZER PARAMETERS FOR THE NORMAL DISTRIBUTION (CONT'D.) 



N = 



128 



65 

66 

67 

68 

69 

70 

71 

72 

73 

74 

75 

76 

77 

78 

79 

80 
81 
82 

83 

84 

85 

86 

87 

88 

89 

90 

91 
S2 

93 

94 

95 

96 

97 

98 

99 
100 
101 
102 

103 

104 

105 

106 
107 
103 

109 

110 
111 
112 

113 

114 

115 

116 
117 
113 

119 

120 
121 
122 

123 

124 

125 

126 

127 

128 



- 0. 000001 

0. 033659 

0. 067331 

0. 101029 
0. 134765 
0. 16e552 
0. 202404 
0. 236333 
0. 270353 
0. 304477 
0. 338720 
0. 373C97 
0. 407620 
0. 442307 
0. 477172 
0. 512231 
0. 547502 
0. 583001 
0. 618747 
0. 654758 
0. 691055 
0. 727658 
0. 764589 
0. 801871 
0.839528 
0. 877587 
0. 916074 
0. 955019 

0. 994453 

1. 034409 
1. 074924 

1. 116037 
1. 157788 
1. 200223 

1. 243391 
1. 237346 
1. 332146 
1. 377G57 
1. 424543 
1. 472299 
1. 521196 
1. 571333 
1. 622833 
1. 675805 
1. 730394 
1. 736759 
1. 845085 
1. 905583 
1. 968503 
2. 034136 
2. 102332 
2. 175010 
2. 251133 
2. 331986 
2. 413222 
2. 510926 
2. 611463 
2. 721700 
2. 844280 
2. 983146 
3. 144589 
3. 339681 
3. 591183 
3. 962315 



0. 016828 
0. 050490 
0. 004172 
0. 117886 
0. 151644 
0. 185460 
0. 219347 
0. 253318 
0. 237387 
0. 321567 
0. 355874 
0. 390320 
0. 424921 
0. 459693 
0. 494651 
0. 529812 
0. 565192 
0. 600810 
0. 636684 
0. 672833 
0. 709277 
0. 746039 
0. 783139 
0. 820602 
0. 858454 
0. 896719 
0. 935428 

0. 974609 
1. 014296 

1. 054523 
1. 095326 
1. 136747 
1. 178823 
1. 221617 
1. 265165 
1. 309527 
1. 354766 
1. 400943 
1. 448148 
1. 496450 
1. 545943 
1. 596733 
1. 648933 
1. 702677 
1. 759111 
1. 815407 
1. 874762 

1. 936404 
2. 000601 
2. 067671 
2. 137993 
2. 212027 
2. 290339 

2. 373634 
2 . 462 ^ 1 1 

2. 559010 

2. 663886 
2. 779511 
2. 909047 
3. 057246 
3. 231933 
3. 447430 

3. 734936 

4. 189694 



ERROR = 0. 000163 

ENTROPY = 6. 699583 
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4. PROGRAM LISTING TO CALCULATE THE LLOYD-MAX QUANTIZER 
PARAMETERS 



C THIS PROGRAM CALCULATES LLOYD-MAX QUANTIZER PARAMETERS BY THE METHOD 
C OF SUCCESSIVE SUBSTITUTION FOR THE NORMAL DISTRIBUTION OF ZERO MEAN 
C AND UNIT VARIANCE 
C The INPUT TO THE PROGRAM IS 

C (1) THE NUMBER OF QUANTIZATION LEVELS N 

C (2) THE MAXIMUM NUMBER OF ITERATIONS M 

C (2) THE ACCURACY AP 

Q A**;*: ick'k'k'k A A***** ***:*:;*:;*: 

REAL*8 X(199) ,T(199) ,XX(199) ,TT(199) ,C ,DELTA, AP( 199) , AP 
1 , ERROR , ENTROP 
INTEGER K ,N , I ,P ,N1 ,N2 ,N3 ,M 
C=DSQRT (00 . 50 DO /DAT AN ( 1 . 0D0 ) ) 

C DO 99 N=110 . 110 

C INPUT THE NUMBER OF QUANTIZATION LEVELS 
N=100 

WRITE(9 ,65)N 
WRITE 9,66) 

WRITE (9, 67) 

WRITE (9 .66) 

C****************:********************************************! if 

C 
C 



INITIALIZATION OF THE QUANTIZER PARAMETERS 

DELTA=0 . 0150D0*DFL0AT (N) 
XX(i)=-10.50000D0 
TT(l)=-5 .50000D0 



=XX a 
)=TT ( 1 ) 

L=2,N 

TT(L)=TT(L-1) -DELTA 
XX(L)=(TT(I)+TT(I-l))/2, 
X(L )=XX(L) 

t(l)=tt(l) 



0D0 



50 CONTINUE 

Q** Ax A Jctc A & A A A A* 

C BEGINING OF THE ITERATIONS 

c M = MAXIMUM NUMBER OF ITERATIONS 

M = 1050 
K=0 

5 K=K+1 

IF (K .GT. M) GO TO 10 
IF (K .GT. 1 )X(N)=XX(N) 

IF (K .GT. 1 T(n)=TT(N 

TT ( 1 )=-C*DEXP ( -XX ( 2 ) *XX (2)/2.0DO)/ (DERFC ( -10.0D0)- 
1DERFC(XX(2)/DSQRT(2 .0D0) ) ; 

T ( 1 ) =TT ( 1 ) 

IF ( N .EQ. 2 ) GO TO 17 
DO 15 P=2 ,N-1 

XX(P)=(T(P)+T(P-1))/2.0D0 

X(P)=XX(P) 

TT(P)=DEXP(-X(P)*X(P)/2.OD0)-DEXP(-X(P+l)*X(P+l)/2.OD0) 
TT(P)=TT(P) *C/ (DERFC(X(P)/DSQRT(2 . 0D0) ) -DERFC (X( P+1 ) /DSQRT(2 . 0 
1 0D0))) 

T(P)=TT(P) 

15 CONTINUE 

17 CONTINUE 

XX(N)=(TT(N)+T(N-1))/2.0D0 

TT(N)=DEXP (-XX(N) *XX(N)/ 2 . 0D0)*C/DERFC(XX(N)/DSQRT(2 .0D0) ) 

X(N)=XX(N) 

t(n)=tt(n) 

N2=IDINT(DFLOAT( (N+2)/2) 

Nl=IDINT(DFLOAT( (N+l)/2 

C CHECKING THE PRECISION OF THE SOLUTION 
c AP = REQUIRED ACCURACY 
AP=0 . 10D-6 
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IF ( (MOD(N, 2) .EQ. 0) .AND. (DABS(X(N2) ) .GT. AP))GO TO 5 
IF ( (MOD(N+l ,2) .EQ. 0).AND. (DABS(T(N1) ) .GT. AP))GO TO 5 
CONTINUE 

10 CONTINUE 

C****************************************************** 

C OUTPUT RESULTS 

IF (MOD(N,2) .EQ. 0)N3=N2 
IF (M0D(N+1 ,2) .EQ. 0)N3=N1 
WRITE (6, 60) K 
DO 120 J=1 ,N3 

IF (J .EQ. 1) 

1 WRITE (9 ,71 )J, T(J) 

IF (J .GT. 1) 

1 WRITE(9,61)J, K(J),T(J) 

120 CONTINUE 

X(N+1)=10.0D0 
X( 1 )=-10 . 0D0 
ERROR=0 . ODO 
ENTROP=0 . ODO 
DO 222 1=1, N 

AP (I )=DERFC (X(I) /DSQRT(2 . ODO) ) -DERFC(X(I+1 ) /DSQRT(2 . ODO ) ) 
AP ( I ) =AP ( I ) / 2 . ODO 
ERROR=ERROR+AP ( I ) *T ( I ) **2 

ENTROP=ENTROP - AP (I)*DLOG(AP(I) )/DL0G(2 .ODO) 

222 CONTINUE 

ERR0R=1 . ODO-ERROR 
WRITE (9, 66) 

WRITE (9, 62) ERROR 
WRITE (9, 66) 

WRITE (9, 63) ENTROP 
WRITE(9,66) 

WRITE (6 , 72) K 
WRITE (9, 90) 

WRITE (9, 66) 

C99 CONTINUE 

65 FORMAT ( 3X , 1 N = ',17) 

66 FORMAT (3X,' ') 

67 FORMAT ( 3X , 1 J X Y ') 

60 FORMAT { IX , 17 , 8( IX, F6 . 4) ) 

90 FORMAT ( 2X , 1 =========================================== 1 ) 

61 FORMAT (IX, 14, 2(2X,F9.6)) 

71 FORMAT ( IX , 14 , 11X, 2(2X,F9.6)) 

62 FORMAT (7X, 'ERROR =', 2(1X,F9.6)) 

72 FORIIAT ( 3X , 1 # ITERATIONS = ',17 ) 

63 FORMAT (7X, 'ENTROPY =' , 2(1X,F9.6)) 

STOP 

END 



135 



LIST OF REFERENCES 



[1.] Lawson, J. "An Elementary Theory of Surveillance," Naval Electronics System 
Command, Technical Report, Aug. 1978. 

[2.] Betsekas, D. P., Tsitsiklis,J. N. and Athans, M., "Convergence Theories of 
Distributed Iterative Process: A Survey," Laboratory of Information and 
Decision Systems, MIT, Report No. LIDS-P-1342, Dec. 1983. 

[3.] Yemini, Y. and Lazer, A., " Towards Distributed Sensor Networks, an Extended 
Abstract," Proceedings of Information Sciences and Systems , 1982, pp. 198-302. 

[4.] Lauer, G., and Sandell, N. R. "Distributed Detection of Known Signal in Noise," 
Report 160, ALPHATECH, Burlington, Mass., Mar. 1982. 

[5.] Tenney, R. R., and Sandell, N. R. "Detection with Distributed Sensors," IEEE 
Trans, on Aerospace and Electronic Systems, vol. ASE-17, 4, Jul. 1981, pp. 
501-510. 

[6.] Van Trees, H. L., Detection, Estimation, and Modulation Theory-Part I, Wiley and 
Sons, New York, 1969. 

[7.] Sarma, V. V. S., and Rao, K. A. Gopala, "Decentralized Detection and 
Estimation in Distributed Sensors Systems," In Proceedings of the IEEE 
Cybernetics and Society Conference, vol-1, 1983, pp. 438-441. 

[8.] Chair, Z., and Varshney, P. K., "Optimal data fusion in multiple sensor detection 
systems," IEEE Transaction on Aerospace and Electronic Systems, vol. AES-22, 1, 
Jan.l9S6, pp. 98-101. 

[9.] Reibman, A. R. and Nolte, L. W. "Optimum Detection and Performance of 
Distributed Sensor System," IEEE Transaction on Aerospace and Electronic 
Systems, vol. AES-23, 1, Jan. 1987, pp. 24-30. 

[10.] Ekchian, L. K., Optimal Design of Distributed Detection Networks, Ph.D. 
Dissertation, MIT, Sep. 1982. 

[11.] Ekchian, L. K. and Tenney, R. R., "Detection Networks," Proceedings of the 21st 
Conference on Decision and Control, 1982, pp. 686-691. 



136 



[12.] Papastavrou, J. D. and Athans, M., "A Distributed Hypothesis-Testing Team 
Decision Problem with Communications cost," Proceedings of the 25st Conference 
on Decision and Control, Athens, Greece, Dec. 1986, pp. 219-225. 

[13.] C. H. Papadimitriou and K. Steigliz, Combinatorial Optimization : Algorithms and 
Complexity, Englewood Cliffs, NJ: Prentice-Hall, 1982. 

[14.] Tsitsiklis, J. N. and Athans, M., "On the Complexity of Decentralized Decision 
Making and Detection Problems," IEEE Trans, on Automatic Control, vol. 
AC-30, No. 5, May 1985, pp. 440-446. 

[15.] Kovatana, T., "Theoretical analysis of instruction alarm using two 
complementary sensors," Final Technical Report, Stanford Research Institute, 
Menlo Park, Calif., Feb. 1973. 

[16.] Fefjar, A., "Combining techniques to improve security in automated entry 
control," In Proceedings of the 1973 Carnahan Conference on Crime 
Countermeasures, May. 1978. 

[17.] Stearns, S. D., "Optimal detection using multiple sensors," Proceedings of 
Carnahan conference on security technology, JJxington, KY, USA, 1983. 

[18.] Max, J., "Quantization for Minimum Distortion," IRE Trans, on Inform. Theory, 
vol. IT-6,1960, pp. 7-13. 

[19.] Pearson ,C. E., Numerical Methods in Engineering and Science, Van Nostrand 
Reinhold Company, New York, 1986, pp. 11-12 and 49-50. 

[20.] Wozencraft, J. M. and Jacobs, I. M., Principles of Communication Engineering, 
Wiley, New York, 1967. 

[21.] E. K. Al-Hussaini and A. M. Al-Bassiouni, "Performance of an ideal switched 
diversity receiver for NCFSK signals with Nakagami fading," Trans. IECE Japan, 
vol. E65, Dec. 1982, pp. 750-751. 

[22.] E. K. Al-Hussaini and A. M. Al-Bassiouni, "Performance of MRC diversity 
Systems for the detection of signals with Nakagami fading," IEEE Trans, on 
Commun. vol. COM-33, No. 12, Dec. 1985, pp. 1315-1319. 

[23.] Lloyd, S. P., "Least Squares Quantization in PCM," Unpublished Bell 
Laboratories manuscript, July, 31, 1957; also in IEEE Trans, on Inform. Theory, 
vol. IT-28, pp. 129-137, 1982. 

[24.] Moose, P M and Al-Bassiouni, A. M "Solution of the Lloyd-Max Quantizer 
Parameters by the Method of Successive Substitution," Naval Postgraduate 
School Technical Report No. NPS 62-88-004. 



137 



[25.] Anthony D. Walen, Detection of Signal in Noise, Academic Press, 1971, p. 366. 

[26.] Sandell, N R., Jr., Variaya, P. C. Athans, M. and Safonov, M. G., "Survey of 
Decentralized Control Methods for Large Scale Systems," IEEE Trans, on 
Automatic Control, vol , AC-23, 1978, pp. 108-128. 

[27.] Yu-Chi Ho and K Al-Ching Chu, "Team Decision Theory and Information 
Structures in Optimal Control Problems- Part I," IEEE Trans, on Automatic 
Control, vol. AC- 17, No. 1, Feb. 1972. 

[28.] Yu-Chi Ho and K Al-Ching Chu, "Team Decision Theory and Information 
Structures in Optimal Control Problems- Part I," IEEE Trans, on Automatic 
Control, vol. AC-17, No. 1, Feb. 1972. 

[29.] Yu-Chi Ho and Marcia P. Kastner, "Teams, Signaling and Information Theory," 
IEEE Trans, on Automatic Control, vol. AC-23, No. 2, April 1978, pp. 305-312. 

[30.] R. Srinivasan, "Distributed Rader Detection Theory," I EE Proceedings, vol. 133, 
F, No. 1, Feb 19S6, pp. 55-60. 

[31.] David A. Castanon and Demosthenis Teneketzis, "Nonlinear Data Fusion," Proc. 
1982 MITjONR Workshop on Command and Control, Monterey, CA, August 
1982, pp. 224-230. 

[32.] G. S. Lauer, D. Teneketzis. D. A. Castanon and N. R. Sandell, Jr. "Recent 
Advances in Distributed Detection," Proc. 1982 MIT/ONR Workshop on 
Command and Control, Monterey, CA, August 1982, pp. 175-179. 

[33.] D. Teneketzis , "The Decentralized Quickest Detection Problem," In proceedings 
of the 21st IEEE Conference of Decision and Control, Fort Lauderdale, FI., 1982, 
pp. 673-679. 

[34.] Sophocles, J. Orfanidis, Optimum Signal Processing, An Introduction, MacMillan 
Publishing Company, 1985, p. 8. 

[35.] Tom M. Apostol, Mathematical Analysis, Addison-Wesley Publishing Company, 
1974, p. 14. 

[36.] David G. Messerschmit, "Quantization for Maximum Output Entropy", IEEE 
Trans, on Infor. Theory, Sep. 1971, pp. 612. 

[37.] Gallagar, R., Information Theory and Reliable Communication, Wiely, 1969. 

[38.] Y. Bar-Shalom, "Tracking Methods in a Multitarget Environment," IEEE Tran. 
Automat, Contr., vol. AC-23, Aug. 1978, pp. 618-626. 



138 



[39.] D. B. Reid, "An Algorithm for Tracking Multiple Targets," IEEE Trans. 
Automat, Contr., vol. AC-24, Dec. 1979, pp. 843-854. 

[40.] Helstrom, C. W., Probability and Stochastic Processes for Engineers, Macmillan 
Publishing Company, 1984, pp. 179-181. 

[41.] Kassam, S. A., "Optimal Quantization for Signal Detection", IEEE, Trans, on 
Communication, vol. COM-25, No. 5, May 1977, pp. 479-484. 

[42.] Wood, C. R., "On Optimum Quantization", IEEE Trans, on Information Theory, 
vol. IT- 15, No. 2, March 1969. 

[43.] Velichkin, A. I., "Optimum Characteristics of Quantizers," Telecommunication 
Radio Eng. (USSR), vol. 18, Feb. 1963, pp.1-7. 

[44.] Garmash, V. A., "The Quantization of Signals with Non-uniform Steps," 
Telecommunications, No. 10, pp. 10-12, Oct., 1957. 



139 



INITIAL DISTRIBUTION LIST 

No. Copies 

1. Defense Technical Information Center 2 

Cameron Station 

Alexandria, VA 22304-6145 

2. Library, Code 0142 2 

Naval Postgraduate School 

Monterey, CA 93943-5002 

3. Department Chairman, Code 62 1 

Naval Postgraduate School 

Monterey, California 93943-5000 

4. Prof. Paul H. Moose, Code 62-Me 5 

Naval Postgraduate School 

Monterey, California 93943-5000 

5. Prof. John M. Wozencraft , Code 62-me 1 

r 6560 Brookdale Dr 

Carmel, California 93943 

6. Prof. Donald P. Gaver Code 55-Gv 1 

Naval Postgraduate School 

Monterey, California 93943-5000 

7. Prof. Harriett B. Rigas 1 

Dept, of Electrical Engineering and System Science 

Michigan State University 
East Lansing. Michigan 48S24 

8. Prof. Arthur L. Schoenstadt Code 53-Zh 1 

Naval Postgraduate School 

Monterey, California 93943-5000 

9. Prof. Charles W. Therrien Code 62-Ti 1 

Na% r al Postgraduate School 

Monterey, California 93943-5000 

10. Prof. Daniel C. Bukofzer Code 62-Bh 1 

Naval Postgraduate School 

Monterey, California 93943-5000 

11. Prof. Roberto Cristi Code 62-Cx 1 

Naval Postgraduate School 

Monterey, California 93943-5000 



140 



12. Prof. Louis L. Scharf 1 

Dept, of Electrical and Computer Engineering 
University of Colorado 
Boulder, CO S0309 

12. Prof. Richard A. .McGonigal Code 0305 1 

Naval Postgraduate School 
Monterey, California 93943-5000 

14. Prof. Donald Trahan Code 53-Tn 1 

Naval Postgraduate School 

Monterey, California 93943-5000 

15. Prof. Richard Franke Code 53-Fe 1 

Naval Postgraduate School 

Monterey. California 93943-5000 

16. Prof. Carroll Wilde Code 53-Wm 1 

Naval Postgraduate School 

Monterey, California 93943-5000 

17. Mr. Larry Frazier code 0141 1 

Naval Postgraduate School 

Monterey, California 93943-5000 

18. Prof. Emad K. Al-Hussaini 1 

EE Dept., Faculty of Engineering 

P.O. 15551, UAE University. Al-Ain 
United Arab Emirates 

19. Prof. Abdel- Wahaab Fayez 1 

Faculty of Engineering, Cairo University 

Giza, Egypt 

20. Prof. P. K. Varshnev 1 

Electrical and Computer Engineering Department 

111 Link Hall. Syracuse University 
Syracuse, NY 13210 

21. Dr. Dave Harris L-205 1 

Lawrence Livermore N'TL Laboratory' 

P.O. Box 808 
Livermore, CA 94550 

22. Chief, Research and Advancement Authority 1 

Egyptian Armed Forces Headquarters 

Nasr City, Cairo, Egypt 

23. Chief, Armament Authority 1 

Armament Authority of Egyptian Armed Forces 

Koprey El-qubaa, Cairo, Egypt 



141 



24. Chief, Training Authority 
Egyptian Armed Forces Headquarters 
Nasr City, Cairo, Egypt 

25. Director, Signal Corps 

Egyptian Armed Forces Headquarters 
Nasr City, Cairo, Egypt 

26. Library, Military Technical College 
Koprey El-qubaa 

Cairo, Egypt 

27. Library, Faculty of Engineering 
Cairo University 

Giza, Egypt 

28. Lieutenant Sarfraz Hussain 
1 577/1 S Samnabad 
Karachi, Pakistan 

29. Mr. Seah Moon Ming 

Blk. IF, Gillman Heights #10-51 
Republic of Singapore 

30. Brigadier General Magdy G. El-Sheimy 

317, King Faesal St., Madkor Station, Pyramid 
Giza, Egypt 

31. Colonel Salah M. El-Magraby 
Building 38, Apt. 21, Tawfeek City. 

Nasr City, Cairo, Egypt 

32. Colonel Abdel-Moniem E. Doma 
45, Ibraheim Abdel-Razik St. 

Ein Shams, Cairo, Egypt 

33. Colonel Mohamed M. Abdel-Moniem 
45, Ibraheim Abdel-Razik St. 

Ein Shams, Cairo, Egypt 

34. Colonel Ismaeel A. Al-Getani 

52, Bochary St., Enterance 2, Apt. 5 
The New, 7(th) Area 
Nasr City, Cairo, Egypt 

35. Colonel Refat M. El-Hefny 
23, High Industrial Institute St. 

Ein Shams (east), Cairo, Egypt 

36. Colonel Abdel Aziz M. Al-Bassiouni 
1 Hamazeen St. 

Kafrel Zaiat, Egypt 



1 



1 



1 



1 



1 



1 



1 



1 



1 



1 



1 



5 



142 



Thesis 

A3347 

c.l 



Thesis 
A3347 
c. 1 



Al-Bassiouni 

Optimum signal process- 
ing in distributed sensor 
systems . 



Al~Bassiouni 

Optimum signal process- 
ing in distributed sensor 
systems. 



